Commandline and GUI synergy

Commandline tools and GUI tools have little or no overlap. Sometimes the GUI model is superior to commandline. For example if you need to decide which images you want to keep and which ones you want to delete. This is tedious via the commandline because you can not see the images. It is tedious via a GUI because you need to shift or tab click to select, then hope your selection is not lost before clicking a few more times to do the delete (and this is the simplest operation basically).

What if there were GUI tools that accepted data via stdin, provided some functionality better suited to a GUI, and then spat out data on stdout after the operation was complete? So what operations are suitable for GUI interaction?

GUI's tend to be mouse focussed. Let's think about mouse input vs keyboard:

Mouse input vs keyboard

Mouse input has a huge range of input values but with less accuracy than keyboard. To illustrate imagine a screen where every pixel was a button that represented some specific input. This would probably allow for a button for all common words, meaning input one word at a time (one word per click). This would be pretty fast if it were possible for the user to a) remember the location of a word on the screen, and b) have the motor control to move the pointer there and click without hitting the wrong button. This is obviously unworkable, but by identifying the weakness in this input we can see the strengths of mouse input. A lot about what sucks with mouse input seems to be indeed where a) you have to memorise a lot of physical locations where clicks need to be directed, and b) when those click locations need to be accurate.

Now think about when mouse input is better than a keyboard: controlling the gun sight in an FPS, or flying a plane in a flight sim. In both those cases there is almost no need to memorise the meaning of a mouse movement because it maps directly to how you might move your head or arm in real life. Also movement does not need to be super precise - a few pixels here or there don't matter a lot. Contrast to keyboard input where you could get a lot of accuracy by tapping keys to rotate a specific number of degrees, but you would get shot or crash before you get there.

Menus in GUI apps are good because they do not require a lot of memorisation - you have categories that can guide you to the button you need to click. Conversely toolbars in GUI's suck because they are often small and you have to memorise what they mean, and by the time you do that it would have been a better investment to memorise a keyboard shortcut.

So what things are keyboards good at. Take drawing a straight line in a painting app. With the mouse this can be a pain. If your using a pencil tool it can be impossible unless you zoom way in to pixel level so your jitter doesn't stray from the intended line too much. Often drawing apps have keyboard shortcuts to ignore one of the mouse axies when dragging. Sometimes you wish for a REPL in the drawing app where you could type "line(10, 10, 100, 10)" rather than try to get the mouse pointer in the exact right place. The Gimp often has text input boxes when you use a tool so you can jump over and explicitely type in certain values.

And obviously the keyboard is superior for writing in a human or computer language. Writing requires a high accuracy. You only need to memorise the location of 30 or so buttons, which takes some time to master but once you do it's transferrable across almost every device.

So input types can be for continuous variables (mouse movement) or discrete variables (key presses). Mouse movement can be used for discrete input (buttons) and keyboards can be used for continous variables (holding a key down). So there is a continuous/discrete axis and an accurate/fuzzy axis:

              |  accurate           fuzzy
--------------|----------------------------------
continuous    |  scroll wheel       mouse axis
              |
discrete      |  key presses        ?
  • Select: presents a list or grid of item from stdin. User toggles items and only those go to stdout.
  • Buckets: user configures buckets in cmdline args. Stdin lines are fed one by one and user clicks the bucket to assign that item to that bucket. Buckets can also have actions like xargs.