Control, Escape, and Meta Tricks

By Susam Pal on 16 Jun 2023

Terminal Tricks

Open a Unix or Linux terminal emulator. If you have on macOS, ensure that the "Use Option as Meta Key" option is enabled in its "Preferences" section. Now type foo bar baz followed by meta+b (i.e., alt+b or option+b on modern keyboards). In a typical desktop environment with a typical and modern terminal emulator running a modern shell like Bash, Zsh, etc., the cursor should move backward by one word. Now type esc b. The cursor should move back again by one word. Finally type ctrl+[ b and the same thing should happen again. How are we able to perform the same operation in three different ways?

Note that if the desktop environment or the terminal emulator or the set of shell key bindings is configured differently, the results may vary. But we will assume that the typical defaults are in effect in the remainder of this post. To understand why these three key sequences yield the same result, it might be a good exercise to run the command cat and type the three key sequences again. The three key sequences we are talking about are:

When we run cat and type the three key sequences mentioned above, the following output may appear:

$ cat

The output shows that the terminal sends the same input to cat each time: the escape character that appears as ^[ in the output and the character b. This becomes more apparent if instead of running cat, we run od -t d1 and type the three key sequences followed by ctrl+d enter:

$ od -t d1
0000000    27  98  27  98  27  98  10                                    

Indeed decimal 27 is the code of the escape character. Similarly decimal 98 is the code of the character b.

Control Codes

Let us first discuss why typing ctrl+[ produces the escape character. The character [ has code 91 (binary 1011011) and holding the ctrl key while typing it results in a control code obtained by taking 91 (binary 1011011), keeping its five least significant bits, and discarding the rest. We get the control code 27 (binary 11011) as the result. This is the code of the escape character. This explains why ctrl+[ produces the escape character and why the escape character is represented as ^[ while typing it into the standard input. The caret sign (^) here is a notation for the ctrl modifier.

The following table provides some more examples of control codes that can be obtained by typing the ctrl key along with some other key.

Key Modified Character Control Character
Binary Decimal Character Binary Decimal Character
ctrl+@ 1000000 64 @ 00000 0 Null
ctrl+g 1000111 71 G 00111 7 Bell
ctrl+h 1001000 72 H 01000 8 Backspace
ctrl+i 1001001 73 I 01001 9 Horizontal Tab
ctrl+j 1001010 74 I 01010 10 Line Feed
ctrl+m 1001101 77 M 01101 13 Carriage Return
ctrl+[ 1011011 91 [ 11011 27 Escape

This explains why typing ctrl+g in a modern terminal emulator produces an audible beep or a visual flash, why ctrl+h erases a character, and so on. This also explains why we can type ctrl+[ in Vim to escape from insert mode to normal mode. While we can type escape with the esc key on the keyboard, we can do so with the ctrl+[ key too within the terminal.

The keen eyed may notice that the table above has lowercase letters in the first column but the second and third columns use the code of the corresponding uppercase letters. The lowercase letters in the first column is merely a notation I am using in this post to identify the keys on a keyboard. They don't actually mean lowercase characters. In fact, the very early keyboards only had uppercase letters and they simply toggled the 7th least significant bit of the modified character to obtain the control code.

By the way, an interesting thing worth noting here is that even if we do consider the code of the lowercase character and pick only its five least significant bits, we get the same control code as we would get if we started with the corresponding uppercase character. For example, consider the key sequence ctrl+g. The uppercase character G has the code 71 (decimal 1000111) and the lowercase character g has the code 103 (decimal 1100111). The five least significant bits are equal in both. So when we pick the five least significant bits and discard the rest, we get the same result, i.e., 7 (binary 111) which is the code of the bell character. This is due to the fact that the five least significant bits of the code of a lowercase character is exactly the same as that of the corresponding uppercase character. They differ only in their sixth least significant bit. This is only an interesting observation. It is of little significance though because like I mentioned earlier, the early keyboards only flipped a bit in the code of the uppercase characters when the ctrl modifier was applied.

In addition to what we have discussed so far, some terminal emulators also implement a few special rules such as the following:

Key Modified Character Resulting Character
Binary Decimal Character Binary Decimal Character
ctrl+space 0100000 32 Space 0 0 Null
ctrl+? 0111111 63 ? 1111111 127 Delete

One could argue that the first row shows a straightforward example of picking the least significiant five bits to arrive at the control code, so it is not really a special case. That is a fair point. However, in the history of computing, different systems have implemented slightly different methods to compute the resulting control codes from our input. Flipping the 7th least significant bit was one of the early methods. Turning off only the the 6th and the 7th least significant bits has been another method. Subtracting 64 from the character code has been yet another one. These different methods produce identical results for the first table but not so for the second table. For example, while turning off the 6th and 7th least significnat bits of 32 (the code of the space character) does give us 0 but merely flipping its 7th bit does not. Further, note that allowing ctrl+space to produce the null character is a bit redundant because ctrl+@ already did that right from the days of very early keyboards. The second entry above is also a special rule because we neither turn off bits nor subtract 64. Instead, we flip the 7th least significant bit which amounts to adding 64 to the code of the modified character. It is also the only control code that has its 6th and 7th least significant bits turned on.

Meta Key Sequences

The meta key no longer exists on modern keyboards. On modern keyboards, we use the alt or option key instead of meta. For example, when a shell's manual says that we need to type meta+b to move the cursor back by one word, what we really type on a modern keyboard is either alt+b or option+b. In fact, the cat and od experiments mentioned earlier show that when we type the modern alternative for the meta key along with another key, what most terminals really send to the underlying program is an escape control code (27) followed by the code of the modified character.

Meta Key Sequence Escape Key Sequence Control Key Sequence
meta+b esc b ctrl+[ b
meta+f esc f ctrl+[ f
meta+/ esc / ctrl+[ /
meta+: esc : ctrl+[ :

There are several layers of software involved between the keyboard input and the application running in the terminal and the exact behaviour of the alt or option key may vary depending on the configuration of each of these layers. Terminal configuration alone is a complex topic that can be discussed extensively. However, the behaviour described here is one of the popular defaults. Alternative behaviours exist but they generally produce similar effects for the user.

Awkward Vim Tricks

Most Vim users know that we can go from insert mode to command-line mode and search for patterns by typing either esc / or C-[ /. But what some people may find surprising is that we can also go from insert mode to searching patterns simply by typing meta+/. Yes, this can be verified by running Vim in a terminal emulator. While insert mode is active, type alt+/ or option+/ and the current mode should instantly switch to the command-line mode with the forward-slash (/) prompt waiting for our search pattern. This works only in a terminal emulator. It may not work in the graphical version of Vim. The table above illustrates why this works in a terminal emulator.

Similarly, in a Vim instance running within a terminal emulator, we can type meta+: to go directly from insert mode to command-line mode and enter Ex commands. We can type meta+0 to go directly from insert mode to the first character of a line, or type meta+$ to go to the end of the line, and so on. These are equivalent to typing esc 0, esc $, etc.

More interestingly, we can type meta+O to open a line above, meta+A to append text at the end of the line, meta+I to append text at the beginning of the line, or meta+S to delete the current line while staying in insert mode! Since the Vim commands O, A, I, and S leave us back in insert mode, we are able to perform an editing operation that involves leaving the insert mode, doing something interesting, and returning to insert mode instantly using the the meta key combination. The following table summarises these observations:

Initial Mode Meta Key Sequence Equivalent To Operation Final State
Insert meta+/ esc / Enter command-line mode to search a pattern Command-line
Insert meta+: esc : Enter command-line mode to enter Ex command Command-line
Insert meta+0 esc 0 Move to the first character of the line Normal
Insert meta+$ esc $ Move to the end of the line Normal
Insert meta+O esc O Begin a new line above and insert text Insert
Insert meta+A esc A Append text to the end of line Insert
Insert meta+I esc I Insert text before the first non-blank in line Insert
Insert meta+S esc S Delete line and insert text Insert

There is no good reason to use Vim like this but it works, thanks to the quirky history of Unix terminals and keyboards!

Comments | #unix | #shell | #technology