Susam's Meetup Pages

From Fill Prefix to TRAMP

2023-12-30T00:00:00Z

Our tiny book club that used to meet during the weekends and holidays and discuss the book Mastering Emacs, 2022 edition concluded today. In our final meeting today, we first discussed how to work across multiple directories in the same Dired buffer. Then we did several demos of the various shells and terminal modes available in Emacs out of the box. That completed our discussion on Chapter 6. Then we moved on to Chapter 7 (the final chapter) that first reiterates the importance of using the describe-system to ask Emacs questions about itself and then offers some recommendations about third-party packages and online Emacs communities. Completing this chapter brought our book club discussions to an end.

A big thanks to Mickey Petersen for writing the book and also very generously granting me the permission to share his book on screen while discussing it.

This book club began on 16 Dec 2022 when we had our first meeting over Jitsi. About 3½ months after beginning these meetings, I posted an update about this book club in another blog post titled From Lunar Phases to Yank-Pop. If you have not read that post yet, I suggest you read it before returning to this post. Especially if you have recently begun learning Emacs, I think you will find that post useful.

Back then, when I posted that last update, we had spent about 26 hours together across 36 meetings and we were reading Chapter 5 of the book. It took another 36 meetings to complete that chapter and the remaining two chapters. After a total of 72 meetings, we completed discussing Chapter 7 of the book today which concluded this series of book club meetings. In total, we have spent a little over 52 hours together to discuss this book, trying out every concept and command introduced in the book and sharing our insights about the material with each other.

In this post, I will share some highlights from our meetings since the last update. These highlights share some concepts and commands we learnt that most members of our book club were not familiar with earlier but were found to be very useful after having learnt them.

Fill Prefix
Elisp Expressions in Replacement Strings
Keep Lines and Flush Lines
Keyboard Macros
DAbbrev
TAB vs M-i
Project Management
Eshell with TRAMP
Thanks

Fill Prefix

Most of us in the book discussion group knew about filling paragraphs with M-q. Consider the following badly formatted paragraphs with very long and very short lines:

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
incididunt ut labore et dolore
magna aliqua.  Arcu dui vivamus arcu felis bibendum ut tristique et egestas.
Bibendum arcu vitae
elementum curabitur vitae.

Now put the point (cursor) anywhere on the paragraph and type M-q. The paragraph gets neatly formatted to something like this:

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua.  Arcu dui
vivamus arcu felis bibendum ut tristique et egestas.  Bibendum arcu
vitae elementum curabitur vitae.

The key sequence M-q invokes the fill-paragraph command that reformats the paragraph such that each line is as long as possible without exceeding the fill width (70 columns by default). Most of us already used this command very often while writing and editing text. However what some of us did not know was that there is such a thing as fill prefix which is taken into account while filling paragraphs. To illustrate this concept, we will first consider this badly formatted paragraph:

:::: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor
:::: incididunt ut labore et dolore
:::: magna aliqua.  Arcu dui vivamus arcu felis bibendum ut tristique et egestas.
:::: Bibendum arcu vitae
:::: elementum curabitur vitae.

Each line has a prefix consisting of four colons and a space. After we reformat this paragraph with M-q, we get something like this:

:::: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
eiusmod tempor :::: incididunt ut labore et dolore :::: magna aliqua.
Arcu dui vivamus arcu felis bibendum ut tristique et egestas.  ::::
Bibendum arcu vitae :::: elementum curabitur vitae.

This is not what we want. We want the paragraph to be formatted such that each line does not exceed 70 characters in length (which we have, in fact, accomplished above) and each line contains the four colons and a space as the prefix (this is broken above). Can we do this? Yes, by setting the fill prefix. Type C-/ or C-x u to undo the bad formatting we did just now and let us try again. This time move the point over to the L of Lorem and type C-x . to set the fill prefix to the current line up to the point. A confirmation is printed in the echo area that ":::: " has been set as the fill prefix. Then type M-q and the paragraph is now neatly formatted to look like this:

:::: Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do
:::: eiusmod tempor incididunt ut labore et dolore magna aliqua.  Arcu
:::: dui vivamus arcu felis bibendum ut tristique et egestas.
:::: Bibendum arcu vitae elementum curabitur vitae.

Note how every line is as long as possible without exceeding 70 characters in length and each line has the fill prefix. Emacs took care to remove the fill prefix from each line, subtract the length of the fill prefix from the maximum character budget it has for each line, reformat the lines and then reinsert the fill prefix on each line of the result.

To turn off the fill prefix, simply set it to an empty prefix by typing C-x . at the beginning of the line. Thus C-a C-x . becomes an idiom for turning off the fill prefix.

Elisp Expressions in Replacement Strings

It was no surprise to anyone in the book discussion group that the key sequence C-M-% f.. RET bar RET starts a search-and-replace operation for strings that match the regular expression pattern f.. to be replaced with the text bar.

The concept of backreferences was also known to most. For example, C-M-% $f..$-$b..$ RET \2-\1 RET searches for strings matching the given regular expression pattern and replaces it with a new string that swaps the positions of the first capturing group and the second capturing group. The backreference \1 refers to the string matched by the first capturing group $f..$ and similarly \2 refers to the string matched by the second capturing group $b..$. In this example, a string like foo-bar is replaced with bar-foo or playful-banter with playban-fulter.

However what came as a surprise to some of us was that we could also use Elisp expressions in the replacement strings. The syntax to do so is to write \, followed by the Elisp expression in the replacement string. For example, consider the key sequence C-M-% f.. RET \,(upcase \&) RET. Note how we are using the backreference \& that refers to the whole match as the argument to the Elisp function upcase that converts its argument to upper-case. This example searches for strings that match the pattern f.. and replaces them with the upper-case form of the match. A string like foo-bar is replaced with FOO-bar.

Here is another slightly more sophisticated example: C-M-% port-$[0-9]+$ RET port-\,(+ 1000 \#1). The backreference \#1 refers to the string matched by the first capturing group $[0-9]+$ as number. The Elisp expression in the replacement pattern simply adds 1000 to that number and replaces the matched string with the result. A string like port-80 becomes port-1080.

Keep Lines and Flush Lines

A nifty set of commands that our group members enjoyed learning were the commands for keeping and flushing lines. These commands can be incredibly useful while filtering large log files. Here is a brief illustration of a couple of these commands:

M-x keep-lines RET f.. RET: Keep lines in region that match the regular expression f.. and delete the rest. If no region is active, then keep matching lines between the point and end of buffer; delete the rest. The deleted lines are not copied to kill ring.
M-x flush-lines RET f.. RET: Delete lines in the region that match the regular expression f... If no region is active, then delete matching lines between the point and end of buffer. The deleted lines are not copied to kill ring.

Note how each point above mentions that the deleted lines are not copied to the kill ring. This can be an inconvenience if we want to quickly yank the deleted lines to another buffer. Emacs 28.1 introduces a couple of more commands that remedy this situation to an extent. Here they are:

M-x copy-matching-lines RET f.. RET: Copy lines in the region that match the regular expression f... If no region is active, then copy matching lines between the point and end of buffer.
M-x kill-matching-lines RET f.. RET: Kill lines in region that match the regular expression f.. to the kill ring. If no region is active, then kill matching lines between the point and end of buffer.

Keyboard Macros

Most experienced Emacs users in our group were aware of keyboard macros. However, some people did learn this wonderful automation feature for the first time in our meetings, so I thought this deserves its own section in this post.

Keyboard macros is a large topic on its own which is best learnt from section Keyboard Macros of the manual. On Emacs, type M-: (info "(emacs) Keyboard Macros") RET to open this section using the Info documentation browser. In this blog post though, we will very briefly talk about keyboard macros that should be enough to get someone very new to it started with them quickly.

Say, we have a buffer that looks like this:

foo:bar:baz
bar:baz:qux
quux:corge:grault
garply:waldo:fred

Now suppose we want to swap the first two fields separated by colon in each line. Of course, we could do it using regular expressions, for example, with the key sequence C-M-% ^$.+?:$$.+?:$ RET \2\1 RET. But we can also solve this problem in a "dumb" way by simply performing the edits necessary to do the swap on one line and then asking Emacs to repeat what we did on the other lines. Here are the steps:

First move the point to somewhere on the first line.
Then type C-x ( to start recording a keyboard macro.
Then type C-a M-d C-d M-f : C-y C-n to swap the first and second fields on the first line and move the point to the next line. This is just one way to achieve the swap. You may use any editing commands you are comfortable with to make the swap happen and move the point to the next line.
Now type C-x ) to stop macro recording.
Now type C-x e to replay the macro in the second line. As soon as we type this key sequence, the swap occurs in the second line and the cursor moves to the third line. Keep repeating this key sequence to keep repeating the swap operation on subsequent lines.

To summarise, C-x ( starts recording a new keyboard macro, C-x ) stops recording the keyboard macro and C-x e replays the last keyboard macro. Alternatively, we could also use the function keys F3 and F4. To start recording a keyboard macro, type F3. Type F4 to stop recording a keyboard macro. Then type F4 again to replay the last macro.

Pay close attention to step 3 above. We start the key sequence with C-a to move the point to the first column. This may feel redundant when the cursor is already at the first column. However in our meetings, I used to emphasise often about the importance of doing this. Typing C-a at the beginning ensures that we do not carry over any assumption about where the cursor is on the line into the rest of the keyboard macro definition we are going to record. By typing C-a, we ensure that no matter where the cursor is on the line, when we replay the macro, the cursor would first move to the beginning of the line. This guarantee allows us to confidently define the rest of the editing operations necessary to perform the swap.

Similarly, at the end we type C-n to move the point to the next line. I used to emphasise the importance of doing this too in our meetings. Moving the cursor to the next line ensures that the cursor is in a good place to allow repeating the keyboard macro again immediately. This is why we could type C-x e (or alternatively, F4) over and over again to replay the macro on subsequent lines. In fact, if we feel confident about the keyboard macro, we can repeat it several times automatically using the digit argument. For example, type C-3 C-x e (or alternatively C-3 F4) to repeat the keyboard macro 3 times. We could also type C-0 C-x e (or alternatively C-0 F4) to repeat the keyboard macro until there is an error (e.g. reaching the end of the buffer).

DAbbrev

DAbbrev stands for dynamic abbrevation. This is a pretty useful package that many of us learnt only from our book club meetings. We discussed two simple key sequences supported by this package:

M-/: Expand the word before the point to the nearest preceding word for which the current word is a prefix. If no suitable preceding word is found, then expand the current word to the nearest succeeding word for which the current word is a prefix.
C-M-/: Find all words in the buffer that has the current word before the cursor as the prefix and expand the current word to the longest common prefix of all these matching words. However, if the longest common prefix of the matching words is same as the word before the cursor, then present them as suggestions for completion. If there is exactly one matching word, expand the word before the cursor to that word.

Let us look at some examples. Suppose there is a buffer with the following one line of text:

abacus apple appliance application

Now if we type ap on the next line and type M-/, DAbbrev automatically expands the partially written word to application because that is the nearest word that has ap as the prefix.

But if we type ap and type C-M-/, the word expands to appl since that is the longest common prefix among all the matching words. If we type C-M-/ again then apple, appliance and application are presented as possible completions in a temporary buffer named *Completions*. If we type ic, so that the word before the cursor becomes applic and type C-M-/, it is expanded to application because that is the only possible completion now.

These two commands are simpler than it sounds from the verbose descriptions of these commands presented in the above paragraphs. When we actually begin to use them, they become intuitive in no time. Roughly speaking, while M-/ expands the word before the point to the nearest preceding word, C-M-/ considers all matching words in the file for expansion and presents completion options to the user when it finds multiple of them.

TAB vs M-i

The behaviour of Emacs when we type TAB can be surprising to beginners. In most other mainstream editors, this key either inserts a tab character or it inserts enough number of spaces so that the cursor moves to the next tab stop. But in Emacs, TAB most often indents the current line according to the syntax rules implemented by the major mode enabled in the buffer.

What is a simple key on other editors happens to be a complex feature in Emacs. The exact behaviour of TAB is controlled by variables like tab-always-indent, indent-line-function, etc. Some major modes may refer to other such special variables to decide what TAB should do. However, as a user of Emacs this is not something we normally have to worry about. Most major modes set up all these variables appropriately, so that TAB almost always does what an experienced Emacs user expects, i.e. indent the current line of code correctly.

But what if we really do want to just insert a tab or enough number of spaces to move the point to the next tab stop column? That is done with M-i. If the variable indent-tabs-mode is set to t, then M-i inserts a literal tab character. If it is set to nil, then M-i inserts enough number of spaces to move the point to the next tab stop column.

To summarise, the behaviour of M-i is similar to the TAB behaviour we observe in other editors. In practice though, the key sequence M-i is rarely required. Most people just type TAB to automatically indent code. In fact, we can also select a region of code and type TAB to reindent the whole region.

Project Management

The project management commands that come out of the box (from the package named project.el) came as a surprise to some. In fact, some members of our group who never used the project management commands earlier happen to use them regularly now after learning about them in our meetings.

When we use a project command like C-x p f to visit a file in the current project, the command automatically detects the top-level directory of the project by checking parent directories for version control system artefacts (e.g. .git directory) and presents files within that top-level directory as autocomplete options.

There is a lot that can be written about the project management features that come out of the box in Emacs. The following list introduces only the very simple ones to get someone started with them:

C-x p f logger RET: Find file with name that matches logger in the current project. This searches all subdirectories recursively. If there is only one matching file (say, src/logger.cc), that file is opened. If there are multiple matching files, they are presented as completion options. Running this command or, in fact, running any project command leads to discovering the current project and adding an entry for the discovered project to ~/.emacs.d/projects. This is useful for a command like the one presented in the next point.
C-x p f foo TAB RET logger TAB RET: When we type C-x p f while visiting a file that does not belong to any project, then its prompts for a project path first. In this example, we type foo TAB RET to automatically expand it to a known project path such as ~/git/foo/ and enter it. Then we type logger TAB RET to automatically expand it to a file name such as src/logger.cc and visit it.
C-x p p bar TAB RET f logger TAB RET: Say we are in project ~/git/foo/ but we want to switch to another previously discovered project ~/git/bar/ and find a file there. To do so, we first type C-x p p to switch project. At the project selection prompt, we type bar TAB to automatically complete the directory path of the known project ~/git/bar/. Then another prompt is presented to choose an action from a number of actions. In this case, we type f to find a file in the project we have switched to. Finally, we type logger TAB RET to automatically expand the partially entered name to a path like src/logger.cc and visit it. The key sequence C-x p p is very useful when the current file belongs to one project but we want to run a project command on another project.
C-x p p ... RET ~/git/baz/ RET f logger TAB RET: This awkward key sequence discovers a new project directory at ~/git/baz/ and then finds a file in it. The key sequence C-x p p ... RET is rarely required though. See the notes after the end of this list to read why.
C-x p g ^key\> RET: Find all matches for the regular expression ^key\> in the current project. The results are displayed in *xref* buffer.
C-x p s: Start a shell in the current project's root directory.
C-x p d: Start Dired in the current project's root directory.
C-x p k yes RET: Kill all buffers belonging to the current project.

There are several more project commands but we will end the above list here for the sake of brevity. Do pay attention to the second point that mentions that if the current file does not belong to any project, we are first prompted to enter the project name. This is a common theme for all project commands. Anytime we invoke a project command, it works on the current project. However if there is no current project, then it automatically prompts us to enter a project name before executing the command.

The key sequence C-x p p ... RET is very rarely required during day-to-day editing activities. Once a project has been discovered (say, due to having run a project command on that project earlier) and added to the list of known projects at ~/.emacs.d/projects, we never have to discover it again. We can use the other key sequences to switch to or work on a known project. Most day-to-day project activities involve working on known projects.

Further, even when we do want to discover a new project and add it to the list of known projects, a much more natural way to do it is to run a project command while we are visiting a file in the project directory. In most cases, we already have a file from some project open in the current buffer. Therefore it makes more sense to just go ahead with a project command, say, with C-x p f, C-x p g, etc. directly instead of explicitly discovering the project with C-x p p ... RET. Merely running a project command while we have a file from a project open ends up discovering the current project automatically. Explicitly discovering projects with C-x p p ... RET is almost never necessary.

Eshell with TRAMP

Many members of our group knew about Eshell and TRAMP separately. For example, M-x eshell RET starts Eshell. Eshell is implemented purely in Elisp and we can use it much like a regular shell. Here is an example session:

~ $ cd /tmp/
/tmp $ echo hello > hello.txt
/tmp $ cat hello.txt
hello
/tmp $ python3 --version
Python 3.11.5
/tmp $ which cd echo cat python3 which
eshell/cd is a byte-compiled Lisp function in ‘em-dirs.el’.
eshell/echo is a byte-compiled Lisp function in ‘em-basic.el’.
eshell/cat is a byte-compiled Lisp function in ‘em-unix.el’.
/usr/bin/python3
eshell/which is a byte-compiled Lisp function in ‘esh-cmd.el’.

We also knew about TRAMP. For example, when we type the key sequence C-x C-f /ssh:alice@box:/tmp/foo.txt RET, TRAMP notices that we intend to connect to a remote system named box as the user alice via SSH and edit a file named /tmp/foo.txt on the remote system. TRAMP then transparently establishes the SSH connection for us. If public key authentication is already set up, then the connection is successfully established immediately. Otherwise it prompts for a password. In the end, we get a buffer to edit the remote file /tmp/foo.txt. Once we have this buffer, we never have to do anything special to work on the remote file. All Emacs commands work seamlessly on this buffer for the remote file. For example, when we type C-x C-s TRAMP would go ahead and save the file to the remote system using the established SSH connection. If we type C-x d, TRAMP would create a Dired buffer for the remote directory /tmp/. All the Emacs commands for working with files and directories we know just work fine with the remote file or directory.

So we knew about Eshell and we knew about TRAMP. However what many of us found pleasantly surprising was how remarkably well Eshell and TRAMP work together. Here is an example Eshell session that illustrates this point:

~ $ cd /ssh:alice@box:/tmp/
/ssh:alice@box:/tmp $ echo foo > foo.txt
/ssh:alice@box:/tmp $ ls
foo.txt
/ssh:alice@box:/tmp $ cd /tmp/
/tmp $ echo bar > bar.txt
/tmp $ ls
bar.txt
/tmp $ cp /ssh:alice@box:/tmp/foo.txt .
/tmp $ ls
bar.txt  foo.txt
/tmp $

Look at how the command cd /ssh:alice@box:/tmp/ above has seamlessly and transparently set the current directory of the shell to the remote directory. When we create a file after that, it gets created on the remote directory! We can work across directories opened with multiple TRAMP methods too. For example first consider this session where the current local user does not have the permissions to write to the local /etc/ directory:

~ $ cp /ssh:alice@box:/etc/wgetrc /etc/
Opening output file Permission denied /etc/wgetrc

But if the current user has sudo privilege, we can do something like this:

~ $ cp /ssh:alice@box:/etc/wgetrc /sudo::/etc/
~ $ ls /etc/wgetrc
/etc/wgetrc

We copied a file from a remote system and wrote it to a protected directory on the local system by using the sudo privilege. We used the ssh method to read a remote file and the sudo method to write the file to a protected local directory. TRAMP really does live up to its name: Transparent Remote Access, Multiple Protocol!

Thanks

It has been a pleasure hosting these Emacs book club meetings throughout this year. I have really enjoyed discussing the book in great detail, examining each new concept introduced in the book carefully and performing demos to illustrate the concepts. A big thank you to the Emacs communities on Libera and Matrix networks who showed interest in these meetings, joined these meetings, participated in the discussions and helped make these meetings successful!

Read on website | #emacs | #meetup | #technology

Notes on Mastering Emacs: Chapter 7: Conclusion

2023-12-29T00:00:00Z

The following notes were taken while discussing Chapter 6 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Third-Party Packages and Tools
Communities

Third-Party Packages and Tools

Here is a list of third-party packages and tools this chapter introduces to the reader:

nov, an EPUB reader
Magit, a Git UI system with a chord-based key system
Multiple Cursors
LSP Mode, a language server interface for Emacs
Eglot, another language server interface for Emacs
Helm, a powerful completion framework
Flycheck, a generic framework for linting and syntax error checking
Org mode, an organiser for notes, agenda, literate programming, etc.
YASnippet, a text snippet expansion tool
Hydra, a package to build flexible popup UIs for key bindings
dumb-jump, a package to jump to definitions from symbols

Note that Eglot comes out of the box since Emacs version 29.1.

Communities

Some of the communities recommended by this chapter are:

Read on website | #emacs | #technology | #book | #meetup

Notes on Mastering Emacs: Chapter 6: The Practicals of Emacs

2023-12-28T00:00:00Z

The following notes were taken while discussing Chapter 6 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Exploring Emacs
Project Management
Xref
Working with Log Files
Highlighting
Auto-Revert Mode
Auto Revert Tail Mode
Browsing Tarballs
Dired: Thumbnail Image Browser
DocView
- DocView Resolution
TRAMP
- Default Directory
- Multi-Hops
EWW: Emacs Web Wowser
Invoking External Browser
Dired
Shell Commands
Compiling in Emacs
Shells in Emacs

Exploring Emacs

The book suggests the following techniques to explore Emacs:

Reading the manual. For example, type M-x info RET or C-h i, then navigate to the Emacs hyperlink, then type C-s version control RET and then navigate to the node named Version Control to read the corresponding manual.

Note that the section named The Info Manual in Chapter 3 offers more alternatives to reach a specific node in a more straightforward manner. For example, C-h i m emacs RET m Version Control RET accomplishes the same result. Alternatively, C-h R emacs RET m Version Control RET also accomplishes the same result. Yet another way to accomplish the same result is to evaluate the Elisp expression (info "(emacs)Version Control"). See section Info in chapter 3 notes for more details.

Yet another way to explore the manual is to use the info-apropos command. For example, type M-x info-apropos RET version control RET to find manuals which have the string "version control" in them.
Using apropos. For example, type C-h d version control RET to search for all symbols whose documentation string contains the specified pattern. Then type C-h a ^vc- RET to search for all commands that match this pattern. This is a convenient way to list the vc commands. Also, see section Apropos in chapter 3 notes for more details.
Exploring prefix keys. For example, type C-x v C-h to list all key sequences bound to the prefix key C-x v. This is in fact a convenient way to list all the vc key bindings. Also, see section Discovering and Remembering Keys in chapter 3 notes for more details.
Describe what a key does. For example, type C-h k followed by C-x v v to see the command that is bound to the latter key sequence as well as its documentation string along with other details like the keymap where the key binding is found, the file where the command is defined, other key bindings for the same command, etc. See section Describe in chapter 3 notes for some more details.
Describe commands. For example, type C-h f vc-dir RET to see information about the vc-dir command. See section Describe in chapter 3 notes for some more details.
Find mode commands. Type C-h m to see the documentation strings of the current major mode and minor modes. A brief summary of the minor modes is shown first, followed by the major mode description. This is followed by documentation strings of the minor modes separated by page breaks (the form feed character that is rendered as ^L in Emacs). See section Describe in chapter 3 notes for some more details.
Type M-X to run execute-extended-command-for-buffer which executes commands that are relevant to the current buffer. While offering completions, it limits the completions to commands relevant to the current buffer. See section M-X: Execute Extended Command for Buffer of chapter 3 notes for more details.

Project Management

Emacs comes with a project management package named project.el which offers commands to operate on projects. When we use a project management command like C-x p f to visit a file in the current project, this package automatically detects the top-level directory of the project by checking parent directories for version control system artifacts (e.g. .git directory) and presents files within that top-level directory as autocomplete options.

The following complete key sequences demonstrate the package management commands mentioned in the book:

C-x p p ... TAB RET ~/git/foo/ RET f README.md: This awkward key sequence discovers a new project directory at ~/git/foo/ and then finds the file named README.md in it. As soon as the key sequence f is typed above, the new project directory is discovered and added to ~/.emacs.d/projects which is where the list of known projects is saved.
C-x p p bar TAB RET f Makefile: Assuming there is already a known project with bar in its name (say, ~/git/bar/) that was discovered earlier, this key sequence switches to that project and finds the file named in Makefile in it.
C-x p f dev/build.sh RET: Find file named dev/build.sh in the current project.
C-x p f build TAB RET: Same as above if dev/build.sh is the only match for build. Otherwise, it presents all files in the current project containing build anywhere in its path name as autocomplete options.
C-x p f bar TAB RET build TAB RET: When we type C-x p f while visiting a file that does not belong to any project, then its prompts for a project name. In this example, we type bar TAB RET to automatically expand it to a known project name such as ~/git/bar/ and enter it. Then we type build TAB RET to automatically expand it to a file name such as dev/build.sh and enter it.

It is worth noting a general point that whenever we invoke a project command while visiting a file that does not belong to a project, the project command prompts for the project name. After we enter the project name, the project command runs on our chosen project. This general point applies to other project commands that come later in this list.
C-x p b Makefile RET: Switch to a buffer named Makefile in the current project. While entering the buffer name when TAB is typed, completion options present buffer names from the current project only.
C-x p k yes RET: Kill all buffers belonging to the current project.
C-x p g ^key\> RET: Find all matches for the regular expression ^key\> in the current project. The matches are found in all files in the project regardless of whether they are currently open in Emacs or not. The matches are displayed in a buffer named *xref*. We can navigate this buffer using key sequences like n, p, etc. Type C-h m in this buffer to see a list of key sequences supported in this buffer. As we navigate this buffer and go from one match to another using n, p, etc. the files containing the match are loaded in a split window automatically with the matching lines automatically centred in that window.
C-x p r ^key\> RET =key= RET: Find all matches for the regular expression pattern ^key\> in the current project and replace them with =key=. The modified files are not automatically saved though. They needed to be saved later explicitly.
C-x p c RET: Compiles the current project. By default, it offers as make -f as the command to be run in the project root. If a specific make target needs to be executed or if another command needs to be executed, then the default command offered may be edited before typing RET.
C-x p v: Runs VC-Dir in the current project's root which in turn shows version control status for the project root.
C-x p s: Start shell in the current project's root directory.
C-x p d RET: Start Dired in the current project's root directory.
C-x p D: Same as above.
C-x p d doc/tutorial/ RET: Start Dired in the doc/tutorial/ subdirectory in the current project.

There are several more project management key bindings. Type C-x p C-h to see a complete list of them.

Xref

Xref provides a generic framework to support commands for cross-referencing in Emacs. While there are several ways to set it up and configure it, the book mentions a particular way to set it up using a couple of external tools. The next two subsections discuss the setup work involved before we can use Xref in a modern way. The remaining subsections discuss how to use Xref.

Xref Setup

By default when we try to look up a definition of an identifier in, say, a C file or Python file, by typing M-., it presents a minibuffer for us to select a tags table file (typically named TAGS). This requires setting up a TAGS file with a tool like ctags. The book, however, does not explore this method for good reason. Typically the TAGS file needs to be created with a tool like ctags or etags for every project we work on. This file contains an index of names found in source code files. We need to periodically update it as the code of our projects evolve, so that this index remains up-to-date. For a long time, this was the only way to maintain an index of the names found in a source code, so that we could perform cross-referencing in editors like Vim and Emacs. Relying on a tool like grep to search the code on the fly was deemed to be quite slow. However, with modern, fast hardware we do not have to work like this anymore. Further, there are search tools like ag and rg which are extremely fast. Given these modern developments, there are simpler ways to set up cross-referencing in Emacs.

The book suggets installing an external package named dumb-jump. It can be installed from MELPA with the key sequence M-x package-install dumb-jump RET. See github.com/jacktasia/dumb-jump for more details about this package. After installing this package, add the following code to the Emacs initialisation file:

(add-hook 'xref-backend-functions #'dumb-jump-xref-activate)

Here is a minimal Elisp code that sets up dumb-jump from scratch and configures it as mentioned above:

(require 'package)
(add-to-list 'package-archives '("melpa" . "https://melpa.org/packages/") t)
(package-initialize)
(unless package-archive-contents
  (package-refresh-contents))
(dolist (package '(dumb-jump))
  (unless (package-installed-p package)
    (package-install package)))
(add-hook 'xref-backend-functions #'dumb-jump-xref-activate)

The above code configures Emacs to use MELPA, retrieve the latest list of packages available there, install dumb-jump from it, as well as set up a hook to activate it automatically when we use certain Xref commands.

Search Tools for Xref

Once Xref is set up with dumb-jump as explained in the previous section, open a source code file (say, a C file or a Python file), move the cursor over to some identifier and type M-. to search that identifier in your environment. By default, it searches for the identifier in files of the same type found under the home directory with a tool like ag, rg or grep (the first one it finds). There is an exception to this rule though. If neither ag nor rg is found and only GNU grep is found, then typing M-. on an indentifier searches the identifier in all files in the home directory (as opposed to searching for files of a specific type). If BSD grep is found instead, then this is not a problem and only files of the current type is searched for the identifier.

Further, while looking up definitions within a Git repository, this package invokes the git grep command to restrict searches to the repository directory.

Let us now look at a few examples of the actual search commands that are executed under the hood when we type M-..

If neither ag nor rg is installed and we only have grep on our system, typing M-. while the cursor is on an identifier named foo in a Python file leads to the execution of a command like this when BSD grep is found:

grep -REn --include '*.py' -e '\s*\bfoo\s*=[^=\n]+' -e 'def\s*foo\b\s*\(' -e 'class\s*foo\b\s*\(?' /Users/susam

If GNU grep is found instead, then all files (not just *.py files) are searched with a command like this:

grep -rEn -e '[[:space:]]*\bfoo[[:space:]]*=[^=\n]+' -e 'def[[:space:]]*foo\b[[:space:]]*\(' -e 'class[[:space:]]*foo\b[[:space:]]*\(?' /home/susam

If rg is the only additional search tool installed, then the following command is executed:

rg --color never --no-heading --line-number -U --pcre2 --type py '\s*\bfoo\s*=[^=\n]+|def\s*foo\b\s*\(|class\s*foo\b\s*\(?' /home/susam

If ag is installed, then the following command is executed:

ag --nocolor --nogroup --python '\s*\bfoo\s*=[^=\n]+|def\s*foo\b\s*\(|class\s*foo\b\s*\(?' /home/susam

When we type M-. in a file that belongs to a Git repository, only the repository directory is searched with a command like this:

git grep --color=never --line-number --untracked -E '\s*\bfoo\s*=[^=\n]+|def\s*foo\b\s*\(|class\s*foo\b\s*\(?' -- /home/susam/repo/*.py

The book makes a mention of rg and remarks about the impressive speed with which it searches the file system. I recommend it too. Since the M-. command may search the whole home directory, if the home directory is very large, having a fast search tool like rg or ag makes a significant difference. For example what could normally take 10 to 20 seconds to search using grep might only take a second or two with rg or ag. I use M-. with rg.

Four Most Common Xref Commands

The book mentions the following commands as the four most common commands we should know about:

M-.: Find definitions of the identifier at point. If a unique definition is found, then the file containing the definition is automatically opened and the definition is centred in the window. If multiple possible candidates are found, then they are displayed in an Xref buffer that we can navigate using key sequences like n or p. As we navigate the Xref buffer, the source of each match is automatically opened in a split window and the matching line is centred.
M-,: Go back to where M-. was last invoked.
M-? foo RET ~/git/foo/ RET: Find all occurrences of the word foo in files of the same type as the current file in the project directory ~/git/foo/. It does not restrict the search to definitions only. If the current file belongs to a project already, then we could simply type M-? foo RET. In fact, since the input to the minibuffer prompt is the identifier at the point by default, we could simply type M-? RET to search for the current identifier in the current project.
C-M-. foo RET: Find symbols matching the given pattern. Although the documentation mentions that this supports regular expressions, it seemed to treat the given pattern as an identifier and searched for that identifier literally. In fact, the rg commands that were executed under the hood were exactly the same as the ones executed by M-.. Thus with dumb-jump enabled, both M-. and C-M-. behave similarly. The only difference is that M-. searches for the identifier at the point whereas C-M-. searches for the identifier we enter at the minibuffer as input.

When multiple cross-references are displayed in the Xref buffer, we can use the following key sequences to work with the Xref buffer.

n: Move to the next cross-reference. The source of the cross-reference is automatically displayed in another window.
n: Move to the previous cross-reference. The source of the cross-reference is automatically displayed in another window.
.: Same as n.
,: Same as p.
RET: Jump to the source of the current cross-reference.
TAB: Hide Xref buffer and jump to the source.
C-o: Show the source of the cross-reference at point in a separate window but keep the point in the Xref window. This is useful when we navigate the Xref buffer using normal Emacs commands like C-p, C-n, C-s, etc. While navigating the Xref buffer with these normal Emacs commands, the source of the cross-references at the point is not automatically displayed. The key sequence C-o helps us to display the cross-reference at the point in this case.
r: Perform search and replace in the names of the references displayed in the Xref buffer. However, I did not find this to be working successfully with dumb-jump. Any attempt to use this command with dumb-jump always led me to the following error: No suitable matches here. This key sequence does work as expected when Xref is invoked from Dired as going to be explained in the next section.

Xref and Dired

Here are some key sequences that demonstrate how we can use Xref with Dired.

C-x d RET: Edit current directory using Dired.
n: Move to the next line. C-n also works.
p: Move to the previous line. C-p also works.
m: Mark the file or subdirectory at the point.
u: Unmark the file or subdirectory at the point.
A f.. RET: Find all matches for the regular expression f.. in the marked files and subdirectories. The matches are always displayed in an Xref buffer, even when a single match is found.
Q f.. RET bar RET: Find all matches for the regular expression f.. in the marked files and subdirectories and replace them with bar.

Working with Log Files

In this section of the book, it discusses a set of commands that are useful for working with log files. Note that some of these commands have been already introduced in the previous chapters. The following list presents the commands discussed in this section of the book:

C-x C-f: Find a file.
C-x C-r: Find file and open in read-only mode.
C-x C-q: Toggle read-only mode.
M-x flush-lines RET b.. RET: Delete lines in region that match the regular expression b... If no region is active, then delete matching lines between the point and end of buffer. The deleted lines are not copied to kill ring. See section Deleting and Keeping Lines of chapter 5 notes for more details.
M-x keep-lines RET b.. RET: Keep lines in region that match the regular expression b.. and delete the rest. If no region is active, then keep matching lines between the point and end of buffer and delete the rest. The deleted lines are not copied to kill ring. See section Deleting and Keeping Lines of chapter 5 notes for more details.
M-s o b.. RET: Show all lines in the current buffer matching the regular expression b... If the region is active, then show matching lines from the region only. The matches are shown in a new Occur mode buffer. The book makes a special mention that we can run M-s o on an Occur mode buffer to filter it further and get the results in another Occur mode buffer. See section Occur Mode in chapter 4 notes for more details.

Highlighting

Section Working with Log Files of Chapter 6 of the book also introduces highlighting commands that can be very useful for highlighting certain strings in the log file. The highlighting commands are demonstrated below with an example.

First create a buffer with the following content.

foo bar baz
Foo Bar Baz
FOO BAR BAZ
foo  bar  baz
Foo  Bar  Baz
FOO  BAR  BAZ

Now type M-s h p f.. SPC b.. RET RET to highlight the phrases matching the regular expression f.. b.. in a case-insensitive and whitespace-insensitive manner. A total of six matches will be highlighted because the first two words and the whitespace between them in all lines match this phrase pattern when we ignore the case of the words and the amount of whitespace. The second RET is meant to accept the default face offered to us for highlighting.
Now type M-s h p b.z RET RET to highlight the phrases matching the regular expression b.z. Again we select the default face offered to us for highlighting. At this point, we should see two sets of highlighting in two different faces.
Now move the cursor to one of the first set of highlights and type M-s h u RET. Those highlights will be unhighlighted. The RET key accepts the default unhighlighting pattern offered to us. It happens to be the pattern with which the highlight under the cursor was highlighted. That is why this key sequence ends up unhighlighting the highlight under the cursor.

If the cursor were not over a highlgiht, then the default unhighlighting pattern offered to us would have been the pattern we used for the last highlight. In that case, we could type M-s h u f.. b.. RET to explicitly specify the unhighlighting pattern.
Now type M-s h u RET again to remove the second set of highlights too.
Type M-s h p F.. SPC B.. RET RET to perform a case-sensitive but whitespace-insensitive highlighting. When there is an uppercase letter in the pattern, the highlighting becomes case-sensitive.
Type M-s h u RET to remove the previous highlighting.
Type M-s h r f.. SPC b.. RET RET to perform a case-insensitive but whitespace-sensitive highlighting. This time, there are only three matches from the first three lines.
Type M-s h u RET to remove the previous highlighting.
Type M-s h r F.. SPC B.. RET RET to perform a case-sensitive and whitespace-sensitive highlighting. The matching strings are found in the second and third lines.
Move the cursor to lowercase bar and type M-s h . to highlight symbol at point. All six occurrences of this symbol are highlighted in a case-insensitive manner because the symbol at point is written in all lowercase.
Move the cursor to Baz and type M-s h . to highlight symbol at point. Only two occurrences of this symbol get highlighted. The highlighted symbols match the symbol Baz exactly (case-sensitive match). The highlighting is done in case-sensitive manner because the symbol at point has at least one uppercase letter.

Auto-Revert Mode

The following steps demonstrate how to use the revert-buffer command and then how to use auto-revert-mode.

In a terminal, run the following command:
```
: > /tmp/log.txt && while true; do date >> /tmp/log.txt; sleep 1; done
```
You could use ansi-term within Emacs too as the terminal if you are familiar with it.
Now within Emacs, type C-x C-f /tmp/log.txt RET.
Wait for a few seconds and type M-x revert-buffer RET yes RET to update the buffer with the latest content of the file from the file system.
Type M-x auto-revert-mode RET to enable automatic update of the buffer as the file changes on the file system. Note that this reloads the entire file whenever a change is detected, so this could be inefficient while working with very large files.
Type M-> to go to the end of the buffer. This moves the cursor to the end of the buffer. Doing this ensures that as the buffer is automatically updated, the cursor automatically keeps moving to the end of the file.
Terminate the command of step 1 and run this command in a terminal:
```
echo hello > /tmp/log.txt
```
The content of the buffer should now automatically truncate and update to just the text hello.
Run the command in step 1 again and confirm that the content of the buffer in Emacs gets updated automatically.
Type M-x auto-revert-mode RET to disable automatic update of the buffer.

Auto Revert Tail Mode

The mode named auto-revert-tail-mode is similar to auto-revert-mode. However, unlike auto-revert-mode which reloads the entire file on every update, the auto-revert-tail-mode only follows the tail of the buffer and appends any new text found to the buffer. The following steps demonstrate this:

Like in the previous section, run the following command:

: > /tmp/log.txt && while true; do date >> /tmp/log.txt; sleep 1; done

Type M-x auto-revert-tail-mode RET. Note that this command follows the tail of the file only. It does not reload the entire file. This can be confirmed with the next step.
Terminate the command of step 1 and run this command in a terminal:
```
echo hello > /tmp/log.txt
```
The buffer for this file in Emacs should automatically update to show the text hello at the bottom. But notice all the earlier text remains intact. The earlier text does not disappear from the buffer because Emacs does not reload the entire file when auto-revert-tail-mode is enabled.
Run the command in step 1 again and confirm that the content of the buffer begins to get updated automatically again.
As of Emacs 28.2, unfortunately running M-x auto-revert-tail-mode RET is not sufficient to disable automatic updates in the buffer. This command does disable the mode but the buffer continues to be updated everytime the file changes. This is very likely a bug in this mode.

As a workaround, disabling auto-revert-mode ends up stopping the auto-update behaviour. There are two ways to do this. You could type M-x auto-revert-mode RET twice: once to enable it and a second time to disable it. Alternatively, just simply type C-0 M-x auto-revert-mode RET which invokes the mode with a prefix argument of zero which ends up disabling the mode.

Browsing Tarballs

The following steps demonstrate how we can not only browse a tarball but also edit files in it and save them back to the tarball.

First, create a directory of text files with the following shell commands:

mkdir -p foo/bar/baz/
echo hello foo > foo/foo.txt
echo hello bar > foo/bar/bar.txt
echo hello baz > foo/bar/baz/baz.txt
tar -caf /tmp/foo.tgz foo/

Confirm that the tarball looks good with these shell commands:
```
tar -tf /tmp/foo.tgz
tar -xOf /tmp/foo.tgz
```
Within Emacs, type C-x C-f /tmp/foo.tgz RET to open the tarball. A list of all entries in the tarball is displayed in a Tar buffer.
Type n and p to navigate the Tar buffer down and up respectively.
With the cursor on the line containing foo/bar/baz/baz.txt, type RET. The content of this entry is now displayed in a new buffer.
Now in the buffer that displays the content of baz.txt, edit its content. Say, type C-a ! to append an exclamation point to this buffer.
Type C-x C-s to save this buffer. This updates the entry of foo/bar/baz/baz.txt within the buffer for foo.tgz. However, the updated tarball is not written to the file system yet.
Type C-x b foo.tgz RET to go back to the buffer with the tarball entry listing.
Finally, type C-x C-s to save the tarball to the file system.
Now repeat step 2. The updated content of foo/bar/baz/baz.txt should now appear in the output.

Dired: Thumbnail Image Browser

Assuming there is a directory ~/foo/ that contains several image files as well as files of other types, the command M-x image-dired RET ~/foo/ RET creates a preview buffer of all images in the directory and displays it along with a normal dired buffer showing the directory listing. Both buffers are displayed in two separate windows.

When the preview buffer is first launched, all image files found in the directory are automatically marked. This can be seen in the Dired buffer. However the preview buffer does not reflect this immediately. Type m in the preview buffer to force it to pick the current list of marked images and highlight them.

As a best practice, remember to type m soon after launching image-dired so that the marked images are accurately displayed in the preview buffer.

Within the preview buffer, the following key sequences are supported:

C-f: Move to next image.
C-b: Move to the previous image.
C-n: Move to next row of images.
C-p: Move to previous row of images.
RET: Display the original image in a display buffer.
m: Mark an image file.
u: Unmark an image file.
d: Flag an image file for deletion.
t t: Tag marked thumbnails. If no thumbnails are marked, tag the current thumbnail.
t r: Remove tag from marked thumbnails. If no thumbnails are marked, remove tag from the current thumbnail.
l: Rotate thumbnail left.
r: Rotate thumbnail right.

Preview Buffer Quirks

The m, u or d commands in the preview buffer are actually meant to mark, unmark or flag the corresponding files in the Dired buffer. The highlighting or unhighlighting that occurs in the preview buffer is merely a convenience feature. The preview buffer may not always accurately reflect the most recent list of all marked and flagged files. Always keep an eye on the Dired buffer to check the most recent state of the files.

Especially, if we go back to the Dired buffer and mark, unmark or flag files, the preview buffer does not reflect it automatically. We need to go to the preview buffer again and perform at least one similar operation (m, u or d) in the preview buffer for it to be updated again. This is why it is important to keep an eye on the Dired buffer to get an accurate account of which files are marked or flagged.

Working on Marked Files

Say we have marked some image files using the key sequence m in the preview buffer. Now we can perform various operations on these marked files. For example, to copy the marked files to /tmp/ directory, in a Dired buffer, type C /tmp/ RET. To move them instead, type R /tmp/.

Deleting Images

When we flag thumbnails by typing d in the preview buffer, the corresponding files are flagged for deletion in the Dired buffer. The first column of the flagged file entries contain the letter D in the Dired buffer. Type x in the Dired buffer to permanently delete (expunge) the flagged files.

Tagging and Untagging

If there are marked images, then the tagging and untagging commands executed in the preview buffer work on those marked images. Otherwise, they work on image corresponding to the current thumbnail. We will refer to these images that the tagging or untagging commands work on as target images in the next few paragraphs..

The key sequence t t trip;oxford;uk RET tags the target images with the tags trip, oxford and uk. The tags must be separated by semicolon as shown in the preceding example. The tags are saved in a path set in the image-dired-db-file variable. Type C-h v image-dired-db-file to read this path. Typically, it is something like ~/.emacs.d/image-dired/.image-dired_db. We will call this the DB file. This file may be manually inspected to see how this command and the next command affect the tags for each thumbnail. Alternatively, type C-t e in a Dired buffer to view and edit the tags of the target files.

The key sequence t r trip RET removes the tag trip from the target images. By virtue of how this functionality is implemented, a key sequence like t r t.*d removes the tags trip;oxford and trip;salford (if present) from the DB file but it does not remove a tag like trip;cambridge (if present).

Using Tags

Tagging thumbnails could be useful if we want to later mark files by tags. In a Dired buffer, the key sequence C-t f t.*d will mark all files whose thumbnails have tags (as they appear in the tags file) matching the regular expression t.*d. For example, images that have with tags trip;oxford;uk as well as trip;london;uk will be marked but images with tags trip;bath;uk and trip;liverpool;uk will not be marked.

Display Buffer

When we type RET in the preview buffer, the original image is displayed in a display buffer. The following key sequences are supported in the display buffer:

s: Resize image to fit window.
f: Display current image in full size.
q: Quit window.

The book makes a note that when we open an image file directly from a Dired buffer, the image is opened in image-mode which is more powerful than the display buffer we get when we open an image from the thumbnail preview window.

DocView

When a PDF or another document of a supported format is opened in Emacs, they are converted to images on the fly and displayed in Emacs. In this section, we will discuss working with PDFs only. The converted images are cached at the directory set in the doc-view-cache-directory variable.

Type C-h v auto-mode-alist RET and search for doc-view in the help buffer to see the list of file formats that Emacs tries to open in DocView.

Ghostscript needs to be installed so that DocView can convert the PDF into images. Further, for some commands where we perform text-based operations on the PDF, we need the pdftotext command so that DocView can extract text from the PDF. Type C-h v doc-view-ghostscript-program RET and C-h v doc-view-pdftotext-program RET to see the external programs that DocView depends on. These programs can be installed with the following command on a Debian or Debian-based Linux distribution:

apt-get install ghostscript poppler-utils

On a macOS system, run the following command instead:

brew install ghostscript poppler

The following list presents some of the key bindings supported by DocView:

n: Go to next page.
p: Go to previous page.
C-x ]: Same as n.
C-x [: Same as p.
SPC: Scroll up if possible or go to next page.
DEL: Scroll down if possible or go to the previous page.
S-SPC: Same as above.
M-<: View the first page.
M->: View the last page.
+: Enlarge the document.
-: Shrink the document.
0: Reset the document size to the initial one.
W: Fit the image width to the window width.
H: Fit the image height to the window height.
P: Fit the image to the window such that neither the document width nor the document height exceed the window width or height respectively.
F: Resize the window so it just fits the page. When there is only window in the frame, the window cannot be resized independently of the frame, so the frame is resized instead.
M-x doc-view-presentation RET: Display document in presentation mode, i.e. as a full screen slide show.

Although not mentioned in the book, here are some commands that show how to perform text-based operations on the PDF. These commands need pdftotext to be installed.

C-s ^f..\> RET: Initiate a new search for lines that begin with a three-lettered word beginning with the letter f. The cursor does not move to the first match automatically. To make the cursor move to the first match, type C-s. This is also explained in the next point.
C-s: When a search has been initiated, jump to the next match for the last search that was initiated.
C-u C-s ^b..\> RET: Initiate a new search. This is useful when a search was already initiated and we want to abandon that search and start another new search.
C-r: Similar to C-s but works in reverse direction. All three commands mentioned above work with C-r too.
C-t: Show tooltip for the current location. Normally, this shows a tooltip like "Page 100 of 314" to describe the current page. When a search is in progress, the tooltip includes all the matches from the current page too.
C-c C-t: Show the current document's content as text. Then type the key sequence C-c C-c to switch to editing the document and C-c C-c again to switch to viewing the document. The key sequence C-c C-c is elaborated a little more in the next point.
C-c C-c: Toggle between editing or viewing the document. In case of PDF, switching to editing the document may not be very helpful because the binary code of the document is opened for editing in this mode which is quite non-trivial to edit directly.

DocView Resolution

If the text in the document looks pixelated in Emacs, set the doc-view-resolution variable to 300 as follows:

(setq doc-view-resolution 300)

This sets the dots per inch resolution used to render the documents to 300. This offers a good trade-off between high quality rendering and fast rendering. After setting this variable, type the following key sequences:

M-x doc-view-clear-cache RET to delete the cache directory.
C-x k to kill the existing DocView buffer (if any).
C-x C-f document.pdf RET to open the document (say document.pdf) again!

Clearing the cache directory and reopening the document in this manner regenerates the images from the documents with the updated resolution.

TRAMP

TRAMP stands for Transparent Remote Access, Multiple Protocol. The general syntax of paths supported by TRAMP is:

/method:[user@][hostname[#port]]:[path]

Here are some complete key sequences that demonstrate various ways to open a remote file using TRAMP:

C-x C-f /ssh:alice@box:~/foo.txt RET: Edit file in a remote host via SSH.
C-x C-f /ssh:susam@box#22:~/foo.txt RET: Same as above. However the port is explicitly specified this time.
C-x C-f /scp:alice@box:~/foo.txt RET: Edit file in a remote host via SCP.
C-x C-f /ssh:box:~/foo.txt RET: Edit file in a remote host via SSH after logging into it with the username of the current user in the current shell.
C-x C-f /ssh:alice@:~/foo.txt RET: Edit file in localhost via SSH after logging into it as a specific user.
C-x C-f /ssh::~/foo.txt RET: Edit file in localhost via SSH after logging into it with the username of the current user in current shell.
C-x C-f /sudo::/etc/hosts: Edit file as superuser.
C-x c-f /sudo:alice@:~/foo.txt RET: Edit file as a specific user.
C-x C-f /sudoedit::/etc/hosts: Edit file as superuser but do not keep an open session running in the background for security reasons. This method has worse performance than the sudo method.
C-x C-f /su::/etc/hosts: Edit file as the root user.
/su:alice@:~/foo.txt: Edit file as a specific user.
/su:alice@localhost:~/foo.txt: Same as above.
C-x C-f /sudo:: RET: Browse the root user's home directory as superuser.
C-x C-f /su:: RET: Browse the root user's home directory with Dired.
C-x C-f /-:: RET: Use tramp-default-method (scp by default) to connect to tramp-default-host (current hostname by default). With the default values of these variables, this leads to connecting to the local system as the current user via SCP and browsing the current user's home directory in Dired.

This chapter recommends looking up the info manual page (tramp) Internal methods but this is very likely an error. For example, evaluating (info "(tramp)Internal methods") leads to the following error:

user-error: No such node or anchor: Internal methods

Instead evaluate (info "(tramp)Inline methods") to reach the correct node that describes the various connection methods.

Default Directory

The variable default-directory is buffer local. Typically, this is automatically set to the directory where Emacs was launched or to the directory of the currently visited file. Commands like C-x C-f defaults to looking up files in this directory.

While editing a remote file via TRAMP, the value for this variable may look something like /ssh:alice@box:/home/alice/. The @ character is displayed in the mode line while editng a remote file.

The chapter presents the following examples of commands that work seamlessly on a remote machine:

C-x d: Manage remote files and directories. We can even copy files (using the key sequence C) between remote and local dired sessions.
M-x compile RET RET: Run make -k (the default) or an arbitrary command remotely. The result is shown in the *compilation* buffer.
M-x rgrep RET f.. RET *.txt RET RET: Use find and grep together to search for the pattern f.. in files matching the pattern *.txt in the current remote directory.
M-x shell RET: Open shell on the remote system in the current remote directory.
M-x eshell RET: Open Eshell on the remote system in the current remote directory.

With Eshell we can go directly into remote directories seamlessly. The following Eshell session illustrates this:

Welcome to the Emacs shell

~ $ uname
Darwin
~ $ cd /ssh:alice@box:~/foo/
/ssh:alice@box:/home/alice/foo $ hostname
debian
/ssh:alice@box:/home/alice/foo $

Multi-Hops

Here are some commands that illustrate how multi-hops work:

C-x C-f /ssh:alice@box|ssh:bob@localhost:~/foo.txt RET: First log in as alice into box and then from there log in as bob into the same system.
C-x C-f /ssh:alice@box|sudo:box:/etc/hosts RET: First log in as alice into box and then edit file as superuser.
C-x C-f /ssh:alice@box|sudo::/etc/hosts RET: Same as above.
C-x C-f /ssh:alice@box|sudo:bob@box:~/bar.txt RET: First log in as alice into box and then use sudo to edit file as bob in the latter user's home directory.
C-x C-f /ssh:alice@box|su::/etc/hosts RET: First log in as alice into box and then edit file as root.
C-x C-f /ssh:alice@box|su:bob@:~/bar.txt RET: First log in as alice into box and then edit file as bob.
C-x C-f /ssh:alice@box|su:bob@box:~/bar.txt RET: Same as above.
C-x C-f /ssh:alice@box1|ssh:bob@box2|ssh:carol@box3:~/foo.txt: An example of two hops.

Bookmarks with key sequences like C-x r m (bookmark-set), C-x r l (bookmark-bmenu-list) and C-x r b (bookmark-jump) work seamlessly for remote files (including multi-hops).

Eshell works seamlessy too across multi-hops. Here is an Eshell session that illustrates it:

~ $ uname
Darwin
~ $ cd '/ssh:alice@box1|ssh:bob@box2|ssh:carol@box3:/home/carol/foo/bar/'
/ssh:alice@box1|ssh:bob@box2|ssh:carol@box3:/home/carol/foo/bar $ uname
Linux
/ssh:alice@box1|ssh:bob@box2|ssh:carol@box3:/home/carol/foo/bar $

EWW: Emacs Web Wowser

The following commands are useful to get started with EWW.

M-x eww RET hello RET: Search for the word "hello" with DuckDuckGo. If the buffer *eww* already exists, then reuse that buffer, otherwise create such a buffer. The search engine can be customised by setting the variable eww-search-prefix (it is "https://duckduckgo.com/html/?q=" by default).
C-u M-x eww RET hello RET: Like previous command except that it creates a new EWW buffer.
M-x eww RET example.net RET: Visit the URL http://example.net. Reuses the *eww* buffer if it exists.
M-x eww RET http://example.net/ RET: Same as above.
C-u M-x eww RET example.net RET: Like before but creates a new EWW buffer.

In the EWW buffer, the following navigation keys work:

TAB: Skip to the next link.
S-TAB or C-M-i: Skip to the previous link.
RET: Browse the URL under point.
C-u RET: Browse the URL under point using an external browser. This is especially useful when we know that the URL we want to open does not render well in EWW.
&: Open the current page in an external browser. Note that unlike the previous command, this command opens the current page. The previous command opens the URL under point instead.
q: Quit EWW.
l: Go to the previously displayed page.
r: Go to the next displayed page.
b: Bookmark the current page.
B: Show bookmarks.
H: Show history of the current EWW buffer. The most recently visited URLs are displayed on top and the oldest ones are shown at the bottom. The current URL is not displayed in the history. Only the older URLs are shown in the history.
R: View the main "readable" parts of the current page. This command uses heuristics to find the parts of the web page that contains the main content and omits the non-content part like navigation menus etc.
M-s M-w: Search the web for the text in the region. If there is no region, then prompt for a search string.
M-RET: Open link in a new EWW buffer.
s: Prompt for an EWW buffer to display and switch to the selected buffer. This is similar to changing to tabs in a desktop web browser.
w: If the point is on a URL or just after a URL, then copy that URL to the kill ring. If the point is at any other place, copy the URL of the current page.

Further, EWW supports a few semantic browsing methods. The pertaining commands are presented below. However note that whether these commands would work on a page or not depends on whether the page provides the relevant navigation aids required by these commands. Here are the key sequences for such commands:

p: Go to the page marked previous. A page is marked previous if there is a <link> tag or an <a> tag for it with the attribute rel="prev" (a standard value for the attribute) or rel="previous" (non-standard value supported by EWW).
n: Go to the page marked next. A page is marked next if there is a <link> tag or an <a> tag for it with the attribute rel="next".
u: Go to the page marked up. A page is marked up if there is a <link> or an <a> tag for it with the attribute rel="up".
t: Go to the page marked top. A page is marked top if there is a <link> tag or an <a> tag for it with the attribute rel="start" or rel="home" or rel="contents".

Invoking External Browser

If we would rather open a URL using our desktop web browser, then we can use the browse-url command like this: M-x browse-url RET http://example.net/ RET.

This command very conveniently picks up the word or domain name at the point or just before the point and uses that as the default value for the URL input. Therefore if the cursor is already on a URL, then we can simply type M-x browse-url RET RET to visit it.

Dired

Dired: Getting Started

There are several ways to start Dired. Some examples are presented below:

C-x C-f ~/foo/bar/ C-d: If IDO or FIDO mode is enabled, then this key sequence automatically opens Dired in the given directory path. Essentially, while using C-x C-f with IDO/FIDO mode, we can type C-d anytime and Dired is opened in the path entered so far.
M-x dired RET ~/foo/bar/ RET: Opens Dired in the given directory path.
M-x dired RET RET: In the previous command, the default input is the path of the current directory (default-directory), so this key sequence conveniently opens Dired in the current directory.
C-x d: Same as above.
C-x 4 d: Like before but open Dired in another window.

The following keys work in a Dired buffer:

RET: Visit the file or directory on the current line.
^: Go up by one directory. If the parent directory is found in an existing buffer, then switch to that buffer. Otherwise create a new buffer to show the parent directory in Dired.
q: Quit Dired window. The buffer remains intact, i.e. the buffer is only buried, not killed.
C-u q: Quit Dired window and kill the buffer.
p or C-p: Move to the previous line and position the point on the filename.
n or C-n: Move to the next line and position the point on the filename.

Note that when we go from one Dired buffer to another (say, by typing RET to enter a subdirectory from a parent directory), then typing q or C-u q buries or kills (respectively) the current buffer and takes us back to the last Dired buffer.

Dired: Marking and Unmarking

The following list describes marking and unmarking commands of Dired:

m: Mark the file or directory at point. If the region is active, mark all files or directories in the region. Note that at least one character of the filename or directory name must lie within the region for a file to be marked.
u: Unmark the file or directory at point. If the region is active, unmark all files or directories in the region. Note that this also removes the flag for deletion (introduced later in this list).
U: Unmark everything. Note that this also removes flags for deletion.
d: Flag the file or directory for deletion. If the region is active, flag all files or directories in the region for deletion.

The following key sequences describe the effects of prefix arguments with marking, unmarking and flagging commands:

C-5 m: Mark 5 files from the current line to 4 more lines below.
C-5 u: Unmark 5 files from the current line for deletion..
C-5 d: Flag 5 files from the current line for deletion.
C-- C-1 m: Mark the file on the previous line.
C-- C-1 u: Unmark the file on the previous line.
C-- C-1 d: Flag the file on the previous line for deletion.
C-- C-5 m: Mark files in the 5 previous lines.
C-- C-5 u: Unmark files in the 5 previous lines.
C-- C-5 d: Flag files in the 5 previous lines for deletion.

Additionally, the chapter mentions the following commands in a separate table but on Emacs 28.2, they seem to have the same effect as one of the commands discussed earlier:

* m: Behaves the same as m and marks files.
* u: Behaves the same as u and unmarks files.

However the following commands (also introduced briefly in the same table but illustrated with complete key sequences below) provide additional marking and unmarking facilities:

* % f.. RET: Mark files with names that match the regular expression f... If the region is active, then only the files in the region that match the pattern are marked. The directories . and .. are never marked.
% m f.. RET: Same as above.
C-u * % f.. RET: Unmark files with names that match the regular expression f... If the region is active, then only the files in the region that match the pattern are unmarked. The directories . and .. are never unmarked.
C-u % m f.. RET: Same as above.
t or * t: Toggle marks. The marked files become unmarked and vice versa. If the region is active, toggle the marks of only the files in the region. The directories . and .. are never toggled. Flagged files are not toggled.
* c * D: Change all files marked with * to be now marked with D (i.e. flagged for deletion). Note that unlike the other commands, this command ignores the active region. It performs the change in the whole buffer.
* c D SPC: Change all files marked with D to be now unmarked.

The chapter also mentions a key sequence * . to mark files by extension but this requires dired-x, so this is discussed in a later section of this page.

Dired: Operations

This section explains some operations we can perform in Dired. If there are one or more items marked in the Dired buffer, then the operations work on the marked items. Otherwise, the operations work on the item under the cursor.

C: Copy marked files or copy the current file. If one file is being copied, this command prompts for the target file path. If multiple files are being copied, this command prompts for the target directory path.
R: Rename marked files or the current file. If one file is being renamed, this command prompts for the target file path. If multiple file are being renamed, this command prompts for the target directory path.
O: Change owner of the marked files or the current file. A complete key sequence may look like O root RET.
G: Change group of the marked files or the current file. A complete sequence may look like G wheel RET.
M: Change the mode of the marked files or the current file. A complete key sequence may look like M u+x RET or M 600 RET. Both symbolic modes like u+x and numeric modes like 600 are supported.
D: Delete marked files, i.e. the files that are marked with * on the leftmost column of the Dired buffer.
x: Delete the files flagged for deletion, i.e. the files that are marked with D on the leftmost column of the Dired buffer.
c: Compress marked files or current file into an archive. The archive file name is prompted. The format of the archive is automatically deduced from the extension of the file name entered at the prompt. A complete key sequence may look like c foo.tar.gz RET or c foo.zip RET.

Note the difference between D and x. The key D deletes marked files but the key x deletes flagged files. Therefore there are two ways of deleting files:

Mark files with m and delete them with D.
Flag files with d and delete them with x.

I normally prefer the second way of deleting files. Since deleting file is a destructive operation which is possibly risky, I like to flag them first with d before deleting them with x. In other words, I flag files for deletions and mark files for everything else. Since I do not use the D key, I can be confident that my marked files are always safe and there is no risk of inadvertently deleting them.

Dired: Copying or Renaming Between Buffers

The following steps explain how we can copy or move files from one Dired buffer to another. First we will see the default behaviour and then we will customise Dired to copy or move files to a particular Dired directory.

Type C-x d /usr/ RET.
Type C-x 2.
Type C-x d /tmp/ RET.
Type C-x 2 agian.
Type C-x d /etc/ RET.
Move the cursor on some file in /etc/ and then type C or R and we will see that the default directory to copy/move the file to is /etc/.
Now type (setq dired-dwim-target t).
Now type C or R again while the cursor is on some file in /etc/. We will see that the default directory to copy/move the file to is /tmp/ now. Since dired-dwim-target is set to non-nil, Dired picks the directory from the next window with a Dired buffer and uses that as the target buffer.

Dired: More Keys

Here are some examples of Dired keys that do not act on marked files but does other interesting work:

g: Refresh the Dired buffer.
+: Create directory. A complete key sequence may look like + bar RET.
s: Toggle sorting by date. By default, the items in the Dired buffer are sorted by file/directory names.
<: Jump to the next directory.
>: Jump to the previous directory.
j: Jump to a file by name. A complete sequence may look like j hosts RET. Note that this only moves the cursor to the line in Dired buffer with the provided filename. It does not visit the file.
M-s a C-s: Perform multi-file incremental search through all marked files or the current file. Marked directories are ignored. Action region is also ignored. It performs the search across all marked files. A complete key sequence may look like M-s a C-s foo and then repeat C-s over and over again to jump through all the matches. When the search reaches the end of one file, the next C-s automatically jumps to the match in the next file.
Q: Perform multi-file regex-based search-and-replace operation through all marked files or the current file. For any marked directories, the search-and-replace operation is performed in all its files recursively. The active region is ignored. A complete key sequence may look like Q f.. RET \&\& RET which searches for strings matching the pattern f.. and duplicates that string. The key sequences supported by C-M-% (query-replace-regexp) like y, n, etc. work here. See section Search and Replace for an account of the supported key sequences. The search results are also displayed in a separate *xref*
A: Find matches for a regular expression pattern in all marked files or the current file. For any marked directories, all its files are searched recursively. The active region is ignored. A complete key sequence may look like A f.. RET. Note that this does not perform incremental search. Instead the search results are displayed in *xref* buffer.
!: Run a shell command on each marked file or directory (or the current file or directory if nothing is marked). The command is executed synchronously. The active region is ignored or the current file or directory. The command works on all marked files and directories. The output is displayed in a separate buffer. If * is present in the command, then each * is replaced with the entire file list and the command runs only once (not multiple times, once for each file). If ? is present in the command, then the command runs multiple times, once for each marked file, with each ? replaced with the name of the file being operated on. It is an error to specify both * and ?. If neither is present, then the command runs multiple times, once for each marked file.
&: Like the previous command but runs the command asynchronously.

To understand the usefulness of g, while Dired is open create a new file in the current directory with, say, C-x C-f foo.txt RET and save it with C-x s. Then kill the buffer for the file with C-x k and return to the Dired buffer. The Dired buffer does not show the new file foo.txt. Now type g to refresh the Dired buffer. As soon as g is typed, the buffer gets updated to display the new file.

To understand the difference between ! and & mark five files and then type the key sequence ! sleep 1; echo. Emacs blocks (i.e. does not react to our keystrokes) for 5 seconds while it runs the given command for each file. When the echo output for all files is obtained after 5 seconds, the output appears and Emacs unblocks again. Now type & sleep 1; echo. Now Emacs remains unblocked while the output of each echo command appears at one second intervals in the output buffer.

Dired-X

Dired-X provides extra Dired functionality. It is not enabled by default. To enable it, add the following line to the Emacs initialisation file:

(require 'dired-x)

The following key sequences are supported by Dired-X:

F: Visit the marked files or the current file. When multiple files are visited, they are opened in split windows distributed as evenly as possible.
C-u F: Visit the marked files or the current file but open them in background, i.e. do not show them on any window.
* .: Mark files with a certain extension. If the region is active, then mark only the files in the region that have the given extension. A complete key sequence may look like * . txt.
! and &: These commands still work the way they were described in the previous section. However with Dired-X enabled, when ! or & is invoked on a single file (either a single marked file or no marked file in which case it operates on the current file), it automatically determines the command to execute for the current file type and offers that as the default input.

Dired: Working Across Directories

The following key sequences offer some support for working across multiple directories in the same Dired buffer:

i: While the cursor is on a line for a directory, it expands the directory listing for that directory in the same Dired buffer. Now we could use the mark, unmark, etc. commands to select files that belong to multiple directories and operate on them from the same Dired buffer.
$: Collapse or expand the current directory listing. If there are multiple directory listings (such as the ones created with i), then move to the next directory listing after collapsing or expanding the current one.

Using i to insert the directory listing of a subdirectory into the current Dired buffer could feel tedious if we want to recursively work on multiple directories. The commands (illustrated with complete key sequences below) may be more suitable for such operations:

M-x find-dired RET RET -name SPC "f*.txt" RET: Find all files and directories in the current directory and its subdirectories recursively with name matching the pattern f*.txt and show the results in the buffer named *Find* with Dired mode enabled in it. Emacs runs the following command to get the results:
```
find . $ -name "f*.txt" $ -ls
```
M-x find-name-dired RET RET f*.txt RET: Same as above. Emacs runs the following command to get the results:
```
find . $ -name f\*.txt $ -l
```
Further on a system with case-insensitive filenames, Emacs is clever enough to use the -iname argument instead of -name so that case-insensitive search is performed.
M-x find-grep-dired RET RET f.. RET: Find all files in the current directory and its subdirectories recursively and list the files where lines matching the regular expression .. is found. The result is shown in a the buffer named *Find with Dired mode enabled in it. Emacs runs the following command to get the results:
```
find . $ -type f -exec grep -q -e f.. \{\} \; $ -ls
```
M-x find-lisp-find-dired RET RET f.. RET: Find all files in the current directory and subdirectories recursively and list the files with names that match the regular expression pattern ... Note that this is different from both find-name-dired and find-grep-dired. The former relies on the Unix find command to match filenames using glob patterns. The latter uses both find and grep to list files that contain a line with a matching regular expression pattern. However this command lists files with names that match a regular expression pattern (not glob pattern). Further this command is implemented purely in Elisp and does not have any external dependencies on tools like find and grep.

Shell Commands

The following complete key sequences demonstrate how we can invoke shell commands from Emacs.

M-! uname RET: Execute shell command and show output.
C-u M-! uname RET: Like above but insert the output into the buffer wherever the cursor is. The cursor remains at the same place. The mark is set to the character just after the last character of the output. Therefore, typing C-x C-x (exchange-point-and-mark) is a quick way to highlight the output just inserted as an active region.
M-! ping SPC -c SPC 4 SPC localhost RET: Execute a slightly long running shell command that takes about 4 seconds to complete. Emacs blocks while the command is running because the command is executed synchronously.
C-u M-! ping SPC -c SPC 4 SPC localhost RET: Like before but the output is inserted into the buffer. Again, Emacs blocks while the command is executed. The output appears in the buffer only after the command completes execution.
M-& ping SPC -c SPC 4 SPC localhost RET: Like M-! but execute shell command asynchronously. Emacs remains unblocked and the output appears in the output buffer as soon as the output is printed by the command.
M-| wc RET: Pipe region to shell command and show output.
C-u M-| wc RET: Pipe region to shell command and replace the region with the output.

The book also mentions that C-u M-& is supposed to work like C-u M-! but asynchronously but I did not find this to be true. For example, C-u M-& uname RET led to the following error Wrong type argument: stringp, (4). This may be a bug in Emacs 28.2.

Compiling in Emacs

The following complete key sequences demonstrate this feature:

M-x compile RET: Runs make -f by default. The default command is offered as a minibuffer input before we type RET. Therefore we can edit the command to any arbitrary command before typing RET.
M-x recompile RET: Runs the last compile command again.
C-x p c: Compile in the current project. See section Project Management for more details.

The compile commands display the output in the *compilation* buffer where the following key sequences work:

M-g M-n: Jump to the next error. The cursor jumps to the next error line in the *compilation* and the source of the matching error line is opened in a separated window.
M-g M-p: Jump to the previous error.
g: Recompile, i.e. run the last compile command again.

Shells in Emacs

M-x shell

The key sequence M-x shell RET starts a shell with input/output done via a buffer. Some important points to keep in mind while using this:

The TAB-completion mechanism of the underlying shell (e.g. Bash, Zsh, etc.) does not work. In fact, TAB invokes Emacs's own completion mechanism.
Programs like top and man that need to control the terminal do not work. Only programs that perform input/output via standard input, standard output and standard error, etc. work well.
Since the shell buffer is made completely of text, all text editing commands of Emacs work seamlessly on the buffer.
We can take the cursor to absolutely anywhere in the buffer and type RET to execute whatever is on that line as a shell command. Shell prompt on the line is automatically excluded from the command to be executed.

Here are some key bindings that work in the shell buffer:

M-p: Cycle backwards through input history.
M-n: Cycle forwards through input history.
C-<up>: Same as M-p. May not work if the desktop environment gobbles up this keystroke.
C-<down>: Same as M-n. May not work if the desktop environment gobbles up this keystroke.
M-r f..: Search history backwards for all commands that match the pattern f... Within the search, we can use incremental search key bindings like C-r, C-s, etc. to search backward, forward, etc. respectively.
C-c C-p: Move to the previous prompt. The cursor moves to the place just after the prompt.
C-c C-n: Move to the next prompt.
C-c C-s out.txt RET: Write output since the last input to a file. Any prompt at the end of the output is not written. Note that by default the output on shell contains the input command as well. The input command is echoed back, so our input command appears twice in the buffer: once where we typed it and once more echoed just before the beginning of the output. This echoed input command is also saved to the file.
C-c C-o: Delete all output since the last input. Any prompt is of course left intact.
C-u C-c C-o: Delete all output since the last input and save it to the kill ring.
C-c C-l: Show the list of recent inputs in the *Input History* buffer.
C-d: If the cursor is at the end of the buffer and there is no input, send EOF. Otherwise delete a character forward.
C-c C-z: Suspend the current job. This performs the same function as C-z in the underlying shell. We can then use job control commands like bg or fg to resume the job as a background process or foreground process.
TAB: Perform completion at point.

M-x ansi-term

The key sequence M-x ansi-term RET RET launches an ANSI-capable terminal emulator. It can run sophisticated programs like top, man, etc. that require terminal capabilities fine. The following key sequences are useful in this terminal emulator:

C-c C-j: Switch to line ("cooked") sub-mode. Emacs editing key sequences work normally in this mode, except RET which sends the current line as a command to the underlying shell.
C-c C-k: Switch to char ("raw") sub-mode. By default, the terminal starts in this mode. Each character we type in this sub-mode is sent directly to the shell, except for the escape character C-c which is used as the prefix keys for the key sequences described in this list.
C-c C-c: Interrupt the current subjob.

M-x eshell

The key sequence M-x eshell RET creates an interactive Eshell buffer if none exists or switches to an existing one. Eshell is implemented in Elisp. It provides Elisp implementation of Unix commands like ls, cp, etc.

The list below provides examples of some commands we can enter directly into Eshell:

which ls: The output should show that ls is an Elisp function.
which which: The output should show that which itself is an Elisp function.
which top: The output should show the file path of the external program top.
ls -l: Run Eshell's implementation of ls written in Elisp.
find-file /etc/hosts: Run the Elisp function named find-file with the argument /etc/hosts thus opening the file in a buffer.
/bin/ls -l: Run the external command ls available provided by the operating system utilities.
python3 --version: Run the external program python3.
top: Start the program top in a separate buffer with term-mode as the major mode.

In the last point we see that for programs like top which need terminal capabilities to show output in a visual fashion (as opposed to just printing output to standard output or standard error), Eshell automatically runs the program in a term-mode buffer, so that the output of the visual program can be handled and displayed correctly. Eshell looks at the list in the variable eshell-visual-commands to determine if a command needs terminal support or not. By default, commands like vi, screen, tmux, top, etc. belong to this list.

Read on website | #emacs | #technology | #book | #meetup

Notes on Mastering Emacs: Chapter 5: The Theory of Editing

2023-09-16T00:00:00Z

The following notes were taken while discussing Chapter 5 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Killing Text
Append Kill
Digit and Negative Arguments
Yanking Text
Yank Pop
Maximum Length of Kill Ring
Killing Lines
Transpose
Filling
Commenting
Search and Replace
Regular Expressions
Changing Case
Counting
Deleting and Keeping Lines
Splitting and Joining Lines
Examining and Fixing Whitespace Issues
Keyboard Macros
Text Expansion
- Abbrev
- DAbbrev
- Hippie Expand
Indenting
- Electric Indentation
- Indenting Current Line
- Use Only Spaces for Indentation
- Tab Width
- Edit Tab Stops
- Indent Region
- Indent Rigidly
Sorting
Aligning
Zap to Char
Zap up to Char
Spell Check
Dictionary Lookup
Quoted Insert
Links

Killing Text

Killing text is equivalent to what we call as cutting text in other editors. Killing some text removes the text from the buffer and adds it to the kill ring. The kill ring is the clipboard of Emacs.

To discover kill commands using the apropos functionality, type C-h a ^kill-.

Here is a list of commands introduced in the first section of this chapter:

C-d: Delete the next character.
<backspace>: Delete the previous character.
M-d: Kill until the end of a word.
C-<backspace>: Kill backward until the beginning of a word.
C-k: Kill the rest of the current line.
M-k: Kill until the end of sentence.
C-M-k: Kill expression following point.
C-S-<backspace>: Kill current line. Does not work in terminal Emacs. Use C-a C-k as alternative.
C-w: Kill text between point and mark.
M-w: Copy text between point and mark to kill ring.
C-M-w: Cause the following command, if it kills, to append to the last stretch of text in the kill ring.
C-y: Yank (paste) the last stretch of text in the kill ring to the buffer.
M-y: Cycle through kill ring.

I have observed that some Emacs users do not bother using M-w to copy to kill ring. Instead they type C-w C-/ to cut the text and immediately undo the cut which effectively leaves the buffer unchanged but inserts a copy of the text that was cut into the kill ring. For example, while C-a C-SPC C-n C-n M-w copies two lines into the kill ring, so does C-a C-SPC C-n C-n C-w C-/. The latter key sequence avoids having to use M-w but it is worth noting that this key sequence does not work in a readonly buffer while the former does. To quickly see the difference, open /etc/hosts as a non-root and non-privileged user and try both the key sequences. The former key sequence does not modify the buffer, so it works perfectly in a readonly buffer. The latter key sequence modifies the buffer when we type C-w, so it does not work in a readonly buffer.

I have also observed that many Emacs users do not bother learning C-S-<backspace> because they can achieve the same results using C-a C-k. Further, C-S-<backspace> does not work in terminal Emacs due to terminal limitations. The key sequence C-a moves the cursor to the beginning of the line and C-k kills everything until the end of the line.

There is a difference between deleting and killing. The first two commands C-d and <backspace> delete characters but the deleted characters are not added to the kill ring. The remaining commands in the list above kill text, i.e. remove the text from the buffer and add it to the kill ring. The killed text can be pasted into the buffer using C-y. For example, M-d M-d M-d C-p C-p C-y kills the next three words and pastes them two lines above.

Append Kill

The key sequence C-M-w is used to ensure that if the next command happens to be a kill command, then the killed text is appended to the last stretch of text in the kill ring.

To understand what this command does we must first understand that after a kill command adds a new stretch of text to the kill ring, subsequent consecutive kills append to the same stretch of text in the kill ring, i.e. consecutive kills form a single large stretch of text in the kill ring. This can be tested by performing consecutive kills and then pasting with C-y. For example, M-d M-d M-d C-p C-p C-y kills 3 words, creates a single stretch of text consisting of those 3 words in the kill ring and pastes that text two lines above.

However, the moment a non-kill command is used, it seals the stretch of text in the kill ring. Any subsequent kill command begins a new stretch of text. For example, M-d M-d M-d C-p M-d M-d C-p C-y kills 3 words at first but then it moves to the previous line sealing that kill text consisting of 3 words. Then it kills 2 words and creates a new stretch of text in the kill ring. Therefore, the final yank command pastes only those 2 words from the kill ring.

This can be a problem if we want to kill text from various parts of the buffer and yet create a single stretch of text that we want to paste somewhere. That's when C-M-w comes useful. For example, M-d M-d M-d C-p C-M-w M-d M-d C-p C-y kills 3 words and creates a single stretch of text in the kill ring consisting of those 3 words. Then it moves one line up and kills 2 more words but this time it appends those 2 words to the existing stretch of text in the kill ring. Finally, it moves two lines up and pastes the entire stretch of text consisting of 5 words into the buffer.

Digit and Negative Arguments

Here are some complete key sequences that demonstrate digit and negative arguments:

M-3 M-d: Kill the next 3 words.
M- M-d : Kill the previous word.
M-- M-3 M-d : Kill the previous 3 words.
C-M-3 C-M-k: Kill the next 3 expressions.
C-M-- C-M-k: Kill the previous expression.
C-M-- C-M-3 C-M-k: Kill the previous 3 expressions.

Yanking Text

There are two key bindings to learn here. The key sequence C-y executes the yank command which yanks the last stretch of text from the kill ring.

In the apropos system, paste is a synonym of yank. Type C-h v apropos-synonyms RET to see all the synonyms define for the apropos system. Thus C-h a paste RET includes the results for yank too.

Yank Pop

The key sequence M-y executes the yank-pop command which replaces a just-yanked kill with an older kill. This key sequence helps us to cycle through the kill ring and fetch older and older kills to be pasted into the buffer.

Here is an experiment to see how we can use C-y and M-y can be used together:

Open a new buffer and type these five words in a single line: foo bar baz qux quux.
Then type C-a M-d C-g M-d C-g M-d. At this point three stretches of text have been inserted into the kill ring. The C-g between every M-d is there to avoid appending kills to the existing stretch of text in the kill ring. This ensures that we have three separate stretches of text in the kill ring.
Now type C-y. The last stretched of kill text, i.e. baz is now pasted into the buffer.
Now without typing any other key sequence, type M-y. The earlier pasted text baz is now replaced with an older stretch of text from the kill ring. Thus baz is replaced with bar.
Now once again type M-y. The earlier pasted text bar is now replaced with a further older stretch of text from the kill ring. Thus bar is replaced with foo.

Note in the previous steps how we are not supposed to type any other key between the first C-y and M-y. Similarly, while cycling through the kill ring, we must not type any other key between the consecutive M-y key sequences. While cycling through the kill ring, when we reach the oldest kill, the next M-y wraps around and brings back the newest kill.

Since Emacs 28, the key sequence M-y also supports browsing the kill ring and yanking any arbitrary entry from the kill ring. For example, after trying the above experiment, type C-g just to make sure that we are breaking any existing C-y or M-y cycle. Then type M-y and a minibuffer prompt appears to yank an arbitrary kill from the kill ring. If we remember the previous kill, we can type it out partially and type TAB to autocomplete it. Alternatively, we could also type TAB initially itself to browse all the kills in the kill ring.

Maximum Length of Kill Ring

Type C-h v kill-ring-max RET to see the maximum length of the kill ring. It is 60 by default.

Killing Lines

Since C-S-<backspace> works only in GUI Emacs and not in terminal Emacs due to terminal limitations, in the section Killing Lines the author recommends installing the package whole-line-or-region which modifies the behaviour of C-w to kill the current line if there is no active region.

This package can be installed with the following command:

M-x package-install whole-line-or-region RET

Then a mode offered by this package can be enabled by adding this line to the Emacs initialisation file:

(whole-line-or-region-global-mode)

After Emacs is started with the updated initialisation file, typing C-w kills the current line if there is no active region. However, if there is an active region then C-w retains the default behaviour of killing the region.

Although the author recommends this package, I do not use this package. I have found C-a C-k to be very effective for killing the current line. However, it is worth noting that for non-empty lines, C-k does not include the newline in the kill by default. If we want to remove the newline too, we must type C-k another time. Therefore, to faithfully reproduce the behaviour of C-w (of whole-line-or-region) or that of C-S-<backspace>, we need to type C-a C-k C-k.

It is possible to change the default behaviour of C-k such that when we type it at the beginning of a line, the trailing newline is included in the kill. To do so, add this to the Emacs initialisation file:

(setq kill-whole-line t)

After Emacs is started with this initialisation file, C-k kills a whole line along with the trailing newline only if cursor is at the start of a line. In other words, with this setting, C-a C-k always kills a whole line along with the trailing newline.

Transpose

Here are some transpose commands:

C-t: Interchange characters around point.
M-t: Interchange words around point.
C-M-t: Interchange expressions around point.
C-x C-t: Exchange current line and previous line.
M-x transpose-paragraphs RET: Interchange current paragraph with next one.
M-x transpose-sentences RET: Interchange the current sentence with the next one.

While using C-t remember that the point is the logical place between two characters. For example if the cursor blinking on the letter e of the word hello, then the point is between the letters h and e. When we type a new character, the new character is inserted where the point is. The key sequence C-t interchanges the characters on both sides of the point, i.e. it exchanges the character the cursor is blinking on with the character just before it.

There is a subtle difference between the way C-x C-t works and the way the other commands work. The other commands exchange the current or previous object with the next one. However, C-x C-t exchanges the current line with the previous one.

Note that the cursor moves to the end of the next object after performing an exchange. This allows the object that moved forward to be dragged further forward by repeated application of the same command. Note again that while the other commands drag the thing at point forward, C-x C-t drags the previous line forward.

If the cursor is on a space between "foo" :: "bar", note that M-t will transpose it to "bar" :: "foo" because it ignores symbols.

Filling

Here are some complete key sequences that perform paragraph filling:

M-q: Refill paragraph.
C-u M-q: Refill paragraph and justify text too.
C-x f 40 RET: Set fill-column to 40.
C-x .: Set the fill prefix to the current line up to point. On performing a fill operation, the fill prefix is inserted at the beginning of every new line created.
C-a C-x .: To cancel the fill prefix, type C-x . at the beginning of a line. Thus C-a C-x . cancels the fill prefix.
M-x auto-fill-mode RET: Toggle auto-filling.

Commenting

Here are some key bindings to add comments to code in various ways:

M-;: Insert or remove comment in a do what I mean (DWIM) fashion. If the line is empty, a comment is inserted at the beginning of the line. If the line is not empty, a comment is inserted at the end of the line and then indented to the column numbered comment-column if it can. If a region is selected, it comments or uncomments that region.
C-x C-;: Comment out or uncomment the current line.
M-x comment-box RET: Comment a region by drawing a box made of comment characters around the selected region. Running this command repeatedly on the same region creates multiple nested comment boxes.
M-j: Insert a new line and continue with the comment if the current line has an open comment. If there is no open comment in the current line, then create a new line and indent.
C-M-j: Same as above.

Here are some variables that control the behaviour of comment-related commands:

comment-style: The default is indent which ensures that new comments created with the comment commands are correctly indented.
comment-styles: An association list with all the available comment styles.
comment-start: String to insert to start a new comment.
comment-end: String to insert to end a new comment.
comment-padding: Extra spacing between the comment characters and the comment text. This is the minimum number of spaces (only if the value of this variable is made of spaces) that Emacs tries to keep between the comment characters and comment text. No spaces are inserted if comment-start and comment-end already provide comment-padding number of spaces or more to separate the comment text.

To demonstrate how changing comment-style changes the commenting behaviour try M-x (setq comment-style 'indent) RET, then select a region and type M-;. The selected region will be commented out with a comment box.

However running M-x (setq comment-style 'aligned) RET, selecting a region in a C buffer and typing M-; does not seem to do anything interesting.

Search and Replace

Here are some complete key sequences that demonstrate search and replace commands:

M-% foo RET bar RET: Replace the string foo with bar while prompting for instruction at every match.
C-M-% f.. RET bar RET: Replace matches for regular expression f.. with bar while prompting for instruction at every match.
M-x query-replace RET foo RET bar RET: Same as M-% foo RET bar RET.
M-x query-replace-regexp RET f.. RET bar RET: Same as C-M-% f.. RET bar RET.
M-x replace-string RET foo RET bar RET: Replace the string foo with bar but do not prompt for instruction at every match.
M-x replace-string RET f.. RET bar RET: Replace matches for regular expression f.. with bar but do not prompt for instruction at every match.

The following key bindings work while a query replace operation is in progress:

y: Replace one match and continue.
SPC: Same as y.
n: Skip to next match.
DEL: Same as n.
q: Exit query replace.
RET: Same as q.
.: Replace one match and exit.
,: Replace and stay at current match.
!: Replace all remaining matches in the buffer with no more questions.
^: Move point back to the previous match.
u: Undo previous replacement.
U: Undo all replacements.
E: Edit replacement string and replace next match.

Just like incremental search (C-s or C-M-s), search and replace performs case folding, i.e. performs case-insensitive match if the search string is a lowercase string. However, the moment we include an uppercase character in the search string, search and replace performs case-sensitive search and replace.

Regular Expressions

This section presents some examples of regular-expression-based search as well as search-and-replace. Here is a simple text buffer where the commands to be presented later can be tried out.

foo-bar-baz
foo-baar-baz
foo-baaar-baz
foo-baaaar-baz
foo-baaaaar-baz
foo-baaaaaar-baz

web
server
webserver
web server
web_server
web->server
web::server
web.server
securewebserver
secure web server
web server port 80

web-server
web-api-server
secure-web-server
web-server-port-80
web-server-port-http
web-server-port-HTTP-80

(1, 2, 3)
[4, 5, 6]
{7, 8, 9}
((10 + 20) * 30)
<40, 50, 60>

"hello, world"
'hello, world'

; comment
# comment
// comment
/* comment */

Here are some complete key sequences that demonstrate regular expressions in search operations:

C-M-s f..: Search for the letter f followed by two characters.
C-M-s foo\|bar: Search for the string foo or bar.
C-M-s ba\{3\}r: Search for the letter b followed by the string aaa and the letter r.
C-M-s ba\{3,5\}r: Search for the letter b followed by 3 to 5 repetitions of the letter a followed by the letter r.
C-M-s port-[0-9]+: Search for the string port- followed by one or more digits.
C-M-s port-[[:digit:]]+: Same as above.
C-M-s port-[[:alnum:]]+: Search for the string port- followed by one or more alphanumeric characters.
C-M-s port-[[:upper:][:digit:]-]+: Search for the string port- followed by consecutive sequence of one or more upper-case letters, digits or hyphen.
C-M-s \<web: Search for the string web at the beginning of a word.
C-M-s web\>: Search for the string web at the end of a word.
C-M-s \<web.+server\>: Search for the string web at the beginning of a word followed by one or more characters and the string server at the end of a word.
C-M-s \_<web.+server\_>: Search for the string web at the beginning of a symbol followed by one or more characters and the string server at the end of a symbol.
C-M-s web\s server: Search for the string web followed by one whitespace character and the string server.
C-M-s web\s-server: Same as above.
C-M-s \s : Search for whitespace character.
C-M-s \s-: Same as above.
C-M-s \sw: Search for word constituent character. Typically uppercase letters, lowercase letters and digits are considered word constituents.
C-M-s \s_: Search for a symbol character that is used in variable names or command names.
C-M-s \s.: Search for punctuation character.
C-M-s \s(: Search for opening pair of a grouping character, e.g. (, [, {.
C-M-s \s): Search for closing pair of a grouping character, e.g. ), ], }, etc.
C-M-s \s": Search for string delimiter. This does not work in text mode but does work in programming modes.
C-M-s \s<: Search for opening comment delimiter. This too does not work in text mode but does work in programming modes.
C-M-s \s>: Search for closing comment delimiter. This too does not work in text mode but does work in programming modes.
C-M-s \Sw: Search for character that is not a word constituent. The pattern \S matches any character whose syntax code is not the given syntax code (w in this example).

All examples above that contain the regular expression \s followed by a character matches a character that belongs to a specific syntax class. For example \s. matches characters that belong to the punctuation syntax class. The syntax class for each character is decided by the current major mode. Thus the same character may belong to different syntax classes in different modes. For example, while the character # belongs to the punctuation syntax class in text mode, it belongs to the comment syntax class in Python mode.

To find out which syntax class a particular character belongs to, place the cursor on the character and type C-u C-x =. The syntax field in the output buffer shows the syntax class of the character.

Here are some complete key sequences that demonstrate various search-and-replace features:

C-M-% $web$$\s-$$server$ RET \3\2\1 RET: Search for the string web followed by a whitespace and the string server and swap web with server.
C-M-% $foo-$\sw+$-baz$ RET \1\?\2 RET: Search for the string foo- followed by a word and the string baz and replace the middle word with text input provided by the user. Before each replace operation, Emacs will prompt the user to edit the replacement pattern by putting the point where \? was in the original replacement string.
C-M-% foo RET \# RET: Search for the string foo and replace each match with an autoincrementing number. The first match is replaced with 0, the second one with 1, the third one with 2 and so on. Precisely speaking, the backreference \# refers to the count of the replacements already made in the current search and replace operation.
C-M-% foo RET \&\& RET: Search for the string foo and duplicate it. The replacement pattern \& stands for the whole match.
C-M-% f.. RET \,(upcase \&) RET: Search for the letter f followed by two characters and replace the match with an uppercase form of the match. The syntax \,(form) is used to evaluate an Elisp form and use its result in the replacement string. The backreference \& refers to the whole match as a string in the Elisp expression.
C-M-% [0-9]+ RET \,(+ 1000 \#&): Search for numbers and add 1000 to each match. The backreference \#& refers to the whole match as a number within the Elisp expression.
C-M-% $\sw+$-$\sw+$ RET \,(upcase \2)-\1 RET: Search for two words separated by a hyphen and then swap them but convert the second word in each match to uppercase. The backreference \2 refers to the string matched by the second capturing group as a string within the Elisp expression.
C-M-% port-$[0-9]+$ RET port-\,(+ 1000 \#1) RET: Search for the string port- followed by a number and add 1000 to the number. The backreference \#1 refers to the string matched by the first capturing group as a string within the Elisp expression.

Changing Case

Here are some commands to change case of text:

M-l: Convert string from point to the end of word to lowercase.
M-u: Convert string from point to the end of word to uppercase.
M-c: Capitalise string from point to the end of word.
C-x C-l: Convert region to lower case.
C-x C-u: Convert region to upper case.
M-x upcase-initials-region RET: Capitalise region.

Note that the commands C-x C-l (downcase-region) and C-x C-u (upcase-region) are disabled by default. Follow the prompts to try it or enable it. A quick way to try it is to type SPC. Also, adding the following to the Emacs initialisation file permanently enables it.

(put 'downcase-region 'disabled nil)
(put 'upcase-region 'disabled nil)

Counting

Here are some commands to count lines, words, characters, patterns, etc:

M-=: Count lines, words and characters in region.
M-x count-words-region RET: Same as above.
M-x count-words RET: Similar to above. If no region is selected, then counts in the entire buffer.
M-x how-many RET f.. RET: Show the number of matches for the regular expression f.. following point. If region is selected, then show the number of matches in the region.
M-x count-matches RET f.. RET: Same as above.

Deleting and Keeping Lines

The commands that are presented in this section can be tested with a buffer like this:

foo
bar

foo
bar

foo
foo

bar
foo



baz
baz
baz

Here are the commands:

M-x delete-duplicate-lines RET: Delete all but one copy of duplicate lines in region. When executed on the whole of the example buffer presented above, it leaves us with three non-empty lines and one blank line. When duplicate lines are encountered, the first instance of each line is kept intact and the others are deleted.
C-u M-x delete-duplicate-lines RET: Like the previous command but search backwards. Thus effectively, the last instance of each repeated line is left intact while the other duplicates are deleted.
C-u C-u M-x delete-duplicate-lines RET: Delete only those duplicate lines that are adjacent to each other. In every contiguous group of duplicate lines, the first one is left intact and the rest are deleted.
C-u C-u C-u M-x delete-duplicate-lines RET: Like the first command in this list but repeated blank lines are left intact. When executed on the whole of the example buffer presented above, it leaves us with three non-empty lines and six blank lines.
M-x flush-lines RET b.. RET: Delete lines in region that match the regular expression b... If no region is active, then delete matching lines between the point and end of buffer. The deleted lines are not copied to kill ring.
M-x keep-lines RET b.. RET: Keep lines in region that match the regular expression b.. and delete the rest. If no region is active, then keep matching lines between the point and end of buffer and delete the rest. The deleted lines are not copied to kill ring.
M-x copy-matching-lines RET b.. RET: Copy lines in region that match the regular expression b.. to the kill ring. If no region is active, then copy matching lines between the point and end of buffer. (Available since Emacs 28.1)
M-x kill-matching-lines RET b.. RET: Kill lines in region that match the regular expression b.. to the kill ring. If no region is active, then kill matching lines between the point and end of buffer. (Available since Emacs 28.1)

To try each command on the entire buffer, first type C-x h to select the entire buffer as the region and then type a command mentioned above.

Splitting and Joining Lines

Here is a list of commands that help with splitting and joining lines:

C-o: Insert a newline after the point but do not move the point.
C-x C-o: On blank line, delete all surrounding blank lines, leaving just one. On isolated blank line, delete the blank line. On non-blank line, delete all consecutive blank lines that follow the non-blank lines. While deleting blank lines it also deletes lines that consist only of whitespaces.
C-M-o: Split current line at the next non-whitespace character after the point while maintaining its indentation. Everything from the next non-whitespace character after the point to the end of the line moves down by one line but the new line is indented so that the column numbers of all the characters that moved down remain the same. If a fill-prefix has been set, say with C-x ., then the fill-prefix is inserted in the new line.
M-^: Join current line with previous line and leave exactly one space between the joined lines. If a fill-prefix is set, say with C-x ., then the fill-prefix is removed while joining lines.

The key sequence C-x C-o is very useful for removing spurious blank lines between paragraphs.

Note that M-^ also works on a region. When a region is active, it joins all lines in the region.

Examining and Fixing Whitespace Issues

Here is a list of commands that are useful in examining whitespace in the current buffer:

M-x whitespace-mode RET: Toggle visualisation of spaces, tabs, newlines and lines longer than whitespace-line-column number of columns (80 by default) with special glyphs and colour.
M-x whitespace-newline-mode RET: Toggle visualisation of newlines.
M-x whitespace-toggle-options RET: Toggle local options for whitespace-mode.

After typing M-x whitespace-toggle-options RET, type a key to tell it what to do. For example, type N and it will start or restart whitespace-mode with the visualisation of newline toggled. Type ? to see the list of all key inputs it supports.

The key sequence M-x whitespace-toggle-options RET may be typed anytime regardless of whether whitespace-mode is currently enabled or not. If whitespace-mode is not enabled, running whitespace-toggle-options automatically enables it. If whitespace-mode is already enabled, then running whitespace-toggle-options and toggling an option, restarts local whitespace-mode with the updated option setting.

Here are some commands to report and clean up whitespace issues:

M-x whitespace-report RET: Shows a report of whitespace issues. The "Current setting" column on left shows the current settings found in the variable whitespace-style. The "Whitespace Problem" column on the right shows the whitespace problems found in the buffer.
M-x whitespace-report-region RET: Like the previous command but reports problems in a region.
M-x whitespace-cleanup RET: Cleans up whitespace issues in the buffer. This command checks the whitespace-style variable to decide which issues to fix. See C-h f whitespace-cleanup RET for complete details.
M-x whitespace-cleanup-region RET: Cleans up whitespace issues in a region. Unlike the previous command, this command does not fix empty lines at the beginning or end of buffer. See C-h f whitespace-cleanup-region RET for complete details.

As mentioned in the list above, the whitespace cleanup functions read the variable whitespace-style to decide which whitespace issues to fix. Say, we do not want to fix trailing whitespace issue but do want to fix other whitespace issues selected by default (e.g. empty lines at the beginning or end of buffer, spaces before tab, etc.), then we need to update the whitespace-style variable as follows:

(setq whitespace-style (delete 'trailing whitespace-style))

Now running whitespace-cleanup or whitespace-cleanup-region is going to skip fixing trailing spaces but it will perform the other cleanups determined by the value of whitespace-style.

Keyboard Macros

The behaviour of keyboard macro key sequences depend on the current context. So they are presented as table below.

Key Command While not recording While recording

F3 kmacro-start-macro-or-insert-counter Start recording Insert counter

F4 kmacro-end-or-call-macro Call macro End recording

C-x ( kmacro-start-macro Start recording Do nothing

C-x ) kmacro-end-macro End recording Do nothing

C-x e kmacro-end-and-call-macro Call macro End recording and call macro

Key	Command	While not recording	While recording
`F3`	`kmacro-start-macro-or-insert-counter`	Start recording	Insert counter
`F4`	`kmacro-end-or-call-macro`	Call macro	End recording
`C-x (`	`kmacro-start-macro`	Start recording	Do nothing
`C-x )`	`kmacro-end-macro`	End recording	Do nothing
`C-x e`	`kmacro-end-and-call-macro`	Call macro	End recording and call macro

The key sequences in the table above can be divided into three groups:

C-x ( and C-x ): These invoke simple commands that start and stop macro recording.
F3 and F3: These are wrappers around the simple commands.
C-x e: This is a slightly high level command too that wraps around simpler macro commands and functions that end recording and calls a macro.

Given these details, there are broadly two ways these macro key sequences can be used. They are shown in the table below.

Operation Using Function Keys Using Control Keys

Start recording F3 C-x (

Stop recording F4 C-x )

Call macro F4 C-x e

Stop recording and call macro F4 F4 C-x e

Repeat call macro F4 e

Operation	Using Function Keys	Using Control Keys
Start recording	`F3`	`C-x (`
Stop recording	`F4`	`C-x )`
Call macro	`F4`	`C-x e`
Stop recording and call macro	`F4 F4`	`C-x e`
Repeat call macro	`F4`	`e`

If you are comfortable using function keys, you might want to follow the second column in the table above. Otherwise, you might want to follow the third column in the table above.

The last row is not mentioned in the book but the fact that e may be used to repeat a macro call performed with C-x e is documented in the Emacs manual: Keyboard Macros: Basic Use.

Note that F3 inserts a counter value and increments the counter value by 1 or by the number specified via a digit argument. Here is an example key sequence that may be typed in a buffer with multiple lines to demonstrate this:

Type C-x ( to start macro recording.
Type C-a F3 . SPC M-c C-n to insert macro counter which 0 by default, followed by dot and space at the beginning of the line, capitalise the first word and move to the next line.
Type C-x e to stop macro recording and call the recorded macro. Now 1, dot and space is inserted at the beginning of the line. The first word of the current line is capitalised and the cursor moves to the next line.
Type e to repeat the macro call. Keep typing e to repeat the macro call.

The behaviour of the the macro commands change with universal arguments and digit arguments as follows:

C-u F3: Execute the last macro, then record new macro and append it to the last macro. Set kmacro-execute-before-append to nil (it is t by default) to prevent executing the last macro before appending a new macro to the last macro.
C-u C-x (: Same as above.
C-5 F3: Start recording but set counter to 5, i.e. while a macro is being recorded, typing F3 inserts 5 the first time, 6 the second time and so on. The numeric prefix argument sets the counter value.
C-5 C-x (: Same as above.
C-u F4: Execute the second macro in the ring.
C-7 F4: Repeat the last macro 7 times.
C-7 C-x e: Repeat the last macro 7 times.
C-0 F3: Repeat macro until there is an error (e.g. reaching the end of a buffer).
C-0 C-x e: Same as above.

Type C-x C-k C-h to discover keyboard macro commands and their key bindings. The list below shows some of the interesting ones mentioned in the book. In the list below, complete key sequences are used, so that they serve as a demonstration of the macro commands.

C-x C-k C-a 20 RET: Add 20 value to the counter.
C-x C-k TAB: Insert counter. Note that we saw earlier that this can also be done with F3.
C-x C-k C-c: Set counter.
C-x C-k C-f %02x: Set macro counter format to zero-padded two-digit hexadecimal numbers with a minimum width of 2.

Note that all of the above commands work fine even when no macro recording is in progress. For example, earlier we saw that F3 inserts the macro counter value only when a macro recording is in progress. However, C-x C-k TAB inserts the macro counter value even when a macro recording is not in progress.

Another interesting feature mentioned in the book is querying for user input while recording a keyboard macro. The key sequence to query the user is C-x C-k q or C-x q. Here are a few complete key sequences that may be used to demonstrate this feature:

F3 C-n C-a foo: C-x q C-e :bar F4: This defines a macro such that when we execute the macro by typing F4 one more time, the macro first inserts foo at the beginning of the next line, then it prompts us to decide if we want to continue with macro execution. If we type type y, then it continues with the remainder of the macro execution. If we type n, it skips the rest of the macro iteration and continue with the next iteration of the macro (such as when we are replaying the macro multiple times with a digit argument). If we type RET, it skips the rest of the macro execution as well as skip any further iterations of the macro (in case we are replaying the macro multiple times).
F3 C-a foo: C-x C-k q C-e :bar F4: Same as above but slightly longer key sequence. The key sequence in the previous point is easier to remember and type.

In the above examples, when the macro playback prompts queries for user input, we can also type C-l to recentre the screen, C-r to enter recursive edit or C-M-c to exit recursive edit.

Note that the key sequence C-l behaves a little differently from the regular C-l. Unlike the regular C-l, successive invocations of this key sequence during macro query does not cause the window to reposition at various places (centre, top and bottom by default) in a cyclical order. Successive invocations of C-l during macro query, leaves the screen centred.

Here are some key sequences to save and recall macros:

C-x C-k C-p: Move to the previous keyboard macro in the keyboard macro ring.
C-x C-k C-n: Move to the next keyboard macro in the keyboard macro ring.
C-x C-k n foo RET: Assign the name foo to the current keyboard macro in the keyboard macro ring. Now the macro can be executed by simply typing M-x foo RET.
M-x insert-kbd-macro foo RET: Insert the definition of the named keyboard macro foo as Elisp code into the current buffer.
C-x C-k b C-c 1: Assign the key sequence C-c 1 to the current keyboard macro in the keyboard macro ring. Now the macro can be executed by simply typing C-c 1.

Finally, here are some commands to edit keyboard macros:

C-x C-k e C-x e: Edit the current keyboard macro.
C-x C-k e M-x foo RET: Edit the keyboard macro named foo.
C-x C-k e C-c 1: Edit the keyboard macro bound to C-c 1.
C-x C-k l: View the most recent 300 keystrokes and edit it to create a new keyboard macro.
M-x kmacro-edit-lossage RET: Same as above.

A few additional commands:

C-h l: See the last 300 characters typed (lossage).
M-x open-dribble-file foo.txt RET: Write input events to a dribble file named foo.txt.
M-: (open-dribble-file nil) RET: Close the dribble file.

Text Expansion

Abbrev

Here are some Abbrev commands:

C-x a l: Take the word before the cursor and define a mode-specific abbreviation for it.
C-x a g: Take the word before the cursor and define a global abbreviation for it.
C-x a i l: Take the abbreviated word before the cursor and define a mode-specific expansion for it.
C-x a i l: Take the abbreviated word before the cursor and define a global expansion for it.

Note that for the expansions to work Abbrev mode should be enabled, say with M-x abbrev-mode RET.

Here are some complete key sequences that demonstrate how we can use Abbrev to define an abbreviation, i.e. text that automatically gets replaced by another text:

Use SPC Debian C-x a l deb RET: Define a mode-specific abbreviation deb such that whenever we type deb, it automatically expands to Debian.
Use SPC Linux C-x a g lnx RET: Define a global abbreviation lnx such that whenever we type lin, it automatically expands to Linux.
Hello SPC wld C-x a i l World RET: Define a mode-specific abbreviation wld such that whenever we type wld, it automatically expands to World.
Hello SPC evry C-x a i g Everyone RET: Define a global abbreviation evry such that whenever we type evry, it automatically expands to Everyone.

Although not mentioned in the book, these commands can be used with a numeric prefix argument to specify the number of words before the cursor to be picked for expansion for the abbreviation we are about to define. Here are some complete key sequences that demonstrate this:

I use Debian GNU/Linux C-3 C-x a l dgl: Define a mode-specific abbreviation dgl such that whenever we type dgl, it automatically expands to Debian GNU/Linux.
I use Debian GNU/Linux C-3 C-x a g dgl: Similar to above but define a global abbreviation.

DAbbrev

There are two key bindings discussed in the book:

M-/: Expand the word just before the cursor to the nearest preceding word for which the current word is a prefix. If no suitable preceding word is found, expand it to the nearest succeeding word for which the current word is a prefix. Repeating this command cycles between the other matches found.
C-M-/: Find all words in the buffer that has the current word before the cursor as the prefix and expand the current word to the longest common prefix of all these matching words. However, if the longest common prefix of the matching words is same as the word before the cursor, then present them as suggestions for completion. If there is exactly one matching word, expand the word before the cursor to that word.

The last command above takes a little while to get used to it. The following steps demonstrate how it works.

Create a text buffer with the following line:
```
abacus apple appliance application
```
Type ap followed by C-M-/, the word expands to appl since that is the longest common prefix among the matching words.
Type C-M-/ again. The matching words apple, appliance and application are presented as possible completions in a temporary buffer named *Completions*.
Now type ic, so that the word before the cursor becomes applic and type C-M-/ again. Now the word before the cursor expands to application because that is the only possible completion now.

Note that by default DAbbrev looks for matching words in other open buffers too and offers them as completions.

Hippie Expand

Unlike DAbbrev, Hippie Expand goes beyond open buffers to look for expansions. The variable hippie-expand-try-functions-list contains a list of expansion functions that hippie-expand uses to look for completions. The book suggests remapping M-/ to invoke hippie-expand with this Elisp code:

(global-set-key [remap dabbrev-expand] 'hippie-expand)

By default, Hippie Expand can complete file names, complete lines, etc. For example, if there is a line for which the current line is a prefix (leading whitespace is ignored while checking for matches), then the current line is expanded to the other matchine line.

Repeated invocations of this command cycles between the matches.

As mentioned earlier, the variable hippie-expand-try-functions-list determines which expansion algorithms are used. Here is an example that demonstrates how we can alter this variable:

(setq hippie-expand-try-functions-list '(try-complete-lisp-symbol))

The above rather unrealistic example severely restricts the expansions Hippie Expand can perform. With the above example, word expansion, line expansion, file name completion, etc. are disabled. Only Elisp symbols are expanded. For example, typing white followed by M-/ first expands the word to whitespace because all matching Elisp symbols have that as the longest common prefix. Typing M-/ over and over again, completes the expansion further with various Elisp symbols.

As mentioned before, the above example is highly atypical. The above example is only meant for demonstrating how this variable can be set. Typically, users add more functions to this variable to add more expansion capabilities.

Indenting

Electric Indentation

Emacs automatically indents code as we type. The major mode decides the automatic indentation behaviour. The automatic identation is provided by a global minor mode named electric-indent-mode which is enabled by default.

Indenting Current Line

Typing TAB indents the current line. In many modes like emacs-lisp-mode, python-mode, text-mode, etc. the command indent-for-tab-command is bound to it. But there are modes that bind another command to TAB. For example, in c-mode, the command c-indent-line-or-region is bound to TAB.

The behaviour of indent-for-tab-command is determined by the variables tab-always-indent. It is t by default which causes TAB to just indent the current line. If set to nil, hitting TAB indents the current line only if the point is before the first non-whitespace character of the line. Otherwise it inserts tabs or spaces to move the point to the next tab stop column. If set to 'complete, typing TAB first tries to indent the current line but if the line is already correctly indented, then it tries to complete the thing at point.

When indent-for-tab-command is bound to TAB and when indent-for-tab-command decides to indent the current line, it calls the function in the variable indent-line-function to perform the indentation. Here is a table that shows what indent-line-function contains in a few major modes where indent-for-tab-command command is bound to TAB:

major-mode indent-line-function

emacs-lisp-mode lisp-indent-line

python-mode python-indent-line-function

text-mode indent-relative

`major-mode`	`indent-line-function`
`emacs-lisp-mode`	`lisp-indent-line`
`python-mode`	`python-indent-line-function`
`text-mode`	`indent-relative`

While lisp-indent-line and python-indent-line attempt to indent the current line according to the syntax of the language, indent-relative inserts tabs and spaces to move the point to the next indentation point where the indentation point is defined as the next non-whitespace character following whitespace. This can be useful in aligning the point with words in the previous line. If the previous line has no indentation point (e.g. the previous line is an empty line or does not have whitespace), then tab-to-tab-stop is invoked which inserts tabs or spaces to move the point to the next tab stop column.

The command tab-to-tab-stop command introduced in the previous paragraph can also be invoked with M-i.

Use Only Spaces for Indentation

By default, Emacs uses a mix of tabs and spaces for indentation and alignment. When it needs to align the first non-whitespace character of a line with a certain token in the previous line, it would insert as many tabs as it can followed by a few spaces if necessary to attain the desired alignment. To force Emacs to always use spaces for indentation and alignment, add the following Elisp code to the Emacs initialisation file:

(setq-default indent-tabs-mode nil)

Tab Width

The variable tab-width is used in various contexts while performing indentation and alignment. For example, when indent-tabs-mode is enabled, for every tab-width columns of indentation required, Emacs inserts a tab to indent the code.

Also, when indent-tabs-mode is set to nil, typing M-i inserts as many spaces as necessary to move the point to the next tab stop column where the distance between two tab stops is assumed to be tab-width.

Edit Tab Stops

The behaviour of M-i can be customised further by manually defining tab stop columns. Type M-x edit-tab-stops RET first. A buffer named *Tab Stops* appears. The second and third line of this buffer contains a ruler to indicate the column numbers. Type : (i.e. colon) in the first line whereever you want to define tab stops. Then type C-c C-c to install the changes. Now when M-i is typed in a text buffer, each time it inserts as many tabs (if indent-tabs-mode is t) or spaces as necessary to move the point to the next tab stop column as defined earlier in the *Tab Stops* buffer.

Indent Region

Typing TAB when a region is active indents the region according to the major mode's indentation rules. It invokes the same command as the one invoked when we type TAB to indent a line. The command bound to it takes care of indenting region. For example, if TAB is bound to indent-for-tab-command, the latter checks if a region is active and if it is, then it simply calls indent-region.

The indent-region command can be invoked explicitly using C-M-\. If fill-prefix has been set, say with C-x ., then it is added to every line in the region being indented. With a numeric prefix argument, each line in the region is indented to the column indicated by the argument. For example, C-M-1 C-M-0 C-M-\ indents each line of the region to column 10.

Indent Rigidly

When we want to rigidly control how a region must be indented, we can type C-x TAB to perform rigid indentation. Doing so allows us to bypass the indentation rules of the major mode. Instead we control exactly how the indentation must be done. The following complete key sequences demonstrates a few examples of rigid indentation:

C-x TAB: Interactively indent region. Type <right> or <left> to increase or decrease indentation by one space respectively. Type <right> or <left> to increase or decrease indentation by one tab stop respectively.
C-6 C-x TAB: Indent region by 6 spaces. Appropriate number of tabs and spaces are inserted to achieve an apparent 6 spaces of indentation. Whether tabs are inserted or not and how many tabs are inserted depend on the values of indent-tabs-mode and tab-width as explained in the previous sections.
C-- C-6 C-x TAB: Reduce indentation of region by 6 spaces.

Sorting

Assuming a region is active, here are some complete key sequences for various sorting commands:

M-x sort-lines RET: Sort lines alphabetically.
C-u M-x sort-lines RET: Reverse sort lines alphabetically.
M-x sort-fields RET: Sort lines alphabetically by the first field alphabetically. Fields are separated by whitespace.
M-2 M-x sort-fields RET: Sort lines alphabetically by the second field.
M-2 M-x sort-numeric-fields RET: Sort lines numerically by the second field.
M-x sort-columns RET: Sort columns between the column position of mark and column position of point.
M-x sort-regexp-fields RET [A-Z]*->$.*$ RET \1 RET: Sort the strings in each line matched by the given regular expression by the field matched by the first (and the only) capturing group in the regular expression. The part of each line that is not matched by the regular expression remains intact. They never move. Only the part of each line that is matched by the regular expression moves around during the sorting operation.
M-x sort-regexp-fields RET: Sort paragraphs alphabetically.

Aligning

The two simple commands for aligning text introduced first in the book are:

M-x align RET: Aligns current region.
M-x align-current RET: Aligns current section. A section is a group of consecutive lines both below, above and including the current line for which the first alignment rule (according to the major mode) applies.

Consider the following Elisp buffer:

(defvar person '(("name" . "Alice")
                 ("city" . "London")
                 ("country" . "UK")))

If we type C-x h followed by M-x align RET or if we put the cursor on any line of the above code and type M-x align-current RET, we get the following result:

(defvar alice '(("name"    . "Alice")
                ("city"    . "London")
                ("country" . "UK")))

There is also an align-regexp command that allows us to parts of lines by regular expressions. The following experiments demonstrate this command.

First create a buffer with the following text:

Alice:London:UK
Bob:Paris:France
Carol:Tokyo:Japan

Type C-x h followed by C-u M-x align-regexp $\s-*$: RET 1 RET 1 RET y RET. The result looks like this:
```
Alice :London :UK
Bob   :Paris  :France
Carol :Tokyo  :Japan
```
Note that the regular expression capturing group $\s-*$ appears as the default in the minibuffer. We only add : to it. Similarly the two occurrences of 1 appear as default values. The first 1 is the default for determining which parenthesis group to modify. The second 1 is the default for amount of spacing to be used during alignment. The y in the end specifies that we want to repeat the alignment throughout the line.
Type C-/ to undo the changes done in the last step. Then type C-x h followed by M-x align-regexp : RET. This is a shorter equivalent to the previous command. The output is same as before.
Type C-/ to undo the changes done in the last step. Then type C-x h followed by C-u M-x align-regexp :$\s-*$ RET 1 RET 1 y RET. Note that the only difference this time is that we place the colon before the parenthesis group. The result looks like this:
```
Alice: London: UK
Bob:   Paris:  France
Carol: Tokyo:  Japan
```
Type C-/ to undo the changes done in the last step. Then type C-x h followed by C-u M-x align-regexp $\s-*$: RET 1 RET 0 y RET. Note that the only difference this time is that we place the colon before the parenthesis group. We specify 0 as the amount of spacing this time, so a minimum of zero spacing is used for alignment when possible. The result looks like the following. Notice the lack of space after Alice and Carol.
```
Alice:London:UK
Bob  :Paris :France
Carol:Tokyo :Japan
```
Type C-/ to undo the changes done in the last step. Then type C-x h followed by C-u M-x align-regexp $\s-*$: RET 1 RET 5 y RET. Note that the only difference this time is that we place the colon before the parenthesis group. We specify 5 as the amount of spacing this time, so a minimum of 5 spaces are used for alignment.
```
Alice     :London     :UK
Bob       :Paris      :France
Carol     :Tokyo      :Japan
```

Zap to Char

The steps below demonstrate the zap-to-char command that is bound to the key sequence M-z. This command kills up to and including the given character.

Create a buffer with the following text:
```
foo bar baz qux quux
```
Type M-< to go to the beginning of the buffer and then type M-z r to kill text up to and including the first occurrence of the letter r. The buffer now looks like this:
```
  baz qux quux
```
Type M-< to go to the beginning of the buffer and then type M-z x to kill text up to and including the first occurrence of the letter x. The buffer now looks like this:
```
 quux
```
Now type C-e SPC C-y to reinsert the killed text at the end of the line. The buffer now looks like this:
```
 quux foo bar baz qux
```
Note that the last step yanks both chunks of text that were killed in the previous two steps. This is due to the fact that consecutive kills append to the same stretch of text in the kill ring. This fact was discussed earlier in section Append Kill
Now type M-< to go back to the beginning of the buffer again. Then type M-2 M-z a. The numeric argument 2 specifies that we want to zap up to the second occurrence of the letter a. The buffer looks like this:
```
z qux
```
Type C-e SPC C-y to reinsert the text killed in the previous step at the end of the line. The buffer looks like this now:
```
z qux quux foo bar ba
```
Type M-- M-2 M-z x to zap backward up to the second occurrence of the letter x. The buffer looks like this now:
```
z qu
```

Zap up to Char

Here are some steps that demonstrate the zap-up-to-char command. This command kills text up to, but not including, the given character.

Create a buffer with the following text:
```
foo bar baz qux quux
```
Type M-< to go to the beginning of the buffer. Then type M-x zap-up-to-char RET b. The result now looks like this:
```
bar baz qux quux
```
Type M-2 M-x zap-up-to-char RET q. The result now looks like this:
```
quux
```
Type C-e SPC C-y to reinsert the killed text at the end of the buffer:
```
quux foo bar baz qux 
```
Type M-- M-2 M-x zap-up-to-char RET b. The buffer now looks like this:
```
quux foo b
```

The author of the book suggests binding this command to M-S-z with the following Elisp code:

(global-set-key (kbd "M-S-z") 'zap-up-to-char)

The above code, however, does not create the binding successfully. Therefore, use the following Elisp code instead:

(global-set-key (kbd "M-Z") 'zap-up-to-char)

Now the commands presented in this section above can be typed as follows:

M-Z b
M-2 M-Z q
M-- M-Z b

Spell Check

The spell checking commands of Emacs require a spell checking program to be installed on the system. Emacs supports the spell checking programs aspell, ispell, hunspell and enchant-2. If multiple programs are present, it looks for them one by one in the order specified in the previous sentence and picks the first one that is found. The following spell checking commands are introduced in the book:

M-$: Check spelling of word under or before the cursor. Possible corrections are offered in a new window. If the word under or before the cursor is already correct, a message like APPLE is correct appears in the echo area. When corrections are offered, each correction is numbered. Type SPC to leave the word unchanged or type a number to choose a numbered correction. Type x to exit the the spelling buffer (the one that shows corrections). Type q to quit the spelling session (kills the spelling program process). To see the list of all key bindings supported, type C-h f ispell-help RET.
M-x flyspell-mode RET: Toggle on-the-fly spell checking. Misspelled words are underlined with squiggly lines. Type C-M-i or C-. to correct a misspelled word under or before the cursor. All possible corrections appear in the echo area. Repeat C-M-i or C-. to cycle through the possible corrections.
M-x flyspell-prog-mode RET: Turns on flyspell-mode for comments and strings only. This is useful while working in a buffer with a programming mode enabled.
M-x ispell-buffer RET: Check the current buffer for spelling errors interactively. Each misspelled word is highlighted and corrections are offered in a new window. The interface and key sequences for making corrections are the same as the ones for M-$ introduced above.
M-x ispell-region RET: Like the previous command but checks the current region for spelling errors.

Dictionary Lookup

The following complete key sequences demonstrate some of the dictionary commands introduced in the book:

M-x dictionary-lookup-definition RET: Look up definitions of the word at or before the cursor.
M-x dictionary-search RET programming RET: Search definitions of the word programming.
M-x dictionary-select-dictionary RET: This command presents a list of available dictionaries in a new buffer. Navigate the buffer and click on a dictionary or type RET while the cursor is on a dictionary entry to select it. For example, move the cursor down to the entry that begins with the text jargon: and type RET and then type M-x dictionary-search RET programming RET to see the definition of the word programming from the Jargon File only.

The dictionary commands first attempt to connect to a locally running dictionary server. If the connection does not succeed, it prompts for consent to connect to dict.org. Typing y at the prompt allows the command to proceed and complete the command.

Quoted Insert

Here are some complete key sequences that demonstrate quoted insert:

C-q C-l: Insert form feed character. This is displayed as ^L in Emacs using the face named escape-glyph. Emacs treats each form feed character as a page break and we can navigate back and forth between pages with C-x [ and C-x ] respectively.
C-q (: Insert a literal open parenthesis. Useful in paredit-mode where the key sequence ( is bound to paredit-open-round which inserts a balanced pair of parentheses. To insert a single parenthesis instead, we can perform a quoted insert with C-q (.
C-q TAB: Insert a literal tab character.
C-q C-j: Insert a literal line feed character, i.e. a newline character.
C-q RET: Insert a literal carriage return. This is displayed as ^M in Emacs.
C-q ESC: Insert a literal escape character. This is displayed as ^[ in Emacs.
C-q C-[: Same as above.

Links

The following list includes some links that were discussed during the book discussion group meetings:

Read on website | #emacs | #technology | #book | #meetup

From Lunar Phases to Yank-Pop

2023-04-02T00:00:00Z

Tiny Book Club

We have a tiny book club that meets every weekend to read and discuss the book Mastering Emacs, 2022 edition written by Mickey Petersen. We go through a few pages of the book every time we meet, do some demos and talk about the concepts we learn from the book. In the 36 meetings that we have had so far, we have spent approximately 26 hours together carefully reading every line of the book, trying out the lessons on an actual editor and experimenting with the new concepts. In the last 3½ months, we have completed four chapters of the book. We are currently reading the fifth chapter. In this post, I'll share what the journey has been like so far and a few interesting things we have learnt.

A big thanks to Mickey Petersen who very graciously granted me the permission to share his book on screen while we discuss the lessons from the book and try out the examples on the editor.

This is the second series of such meetings I have been hosting. The first one was about analytic number theory that began in March 2021. It ran for seven months and finally concluded in October 2021 after 120 meetings. I chose Emacs as the topic for the next series. The book Mastering Emacs by Mickey Petersen seemed like a great choice for it.

Our new discussion group for Emacs began on 16 Dec 2022. We have been meeting over Jitsi during the weekends. Each meeting is approximately 40 minutes long. With my desktop shared via Jitsi, I demonstrate all the concepts we find in the book in my Emacs editor. On an average, we see about 7 participants in each meeting. Some participants are regulars who join the meetings every weekend, follow the lessons, share their comments, etc. It has been a fun experience so far.

Tiny Book Club
Variety of Professions
Lunar Phases
Returning to Mark
Working With Other Windows
Tab Bars
Back to Indentation
Exchange Point and Mark
Search Toggles
Word Search Mode
Occur Mode
Imenu
Helm
Appending Kills
Yank-Pop
Join Us

Variety of Professions

Most members of our discussion group come from the #emacs channels of Libera and Matrix networks. An interesting thing I noticed in our Emacs book discussion group is that we have a good mix of members from diverse backgrounds. In the previous series of meetings on analytic number theory, almost every participant had a career in software engineering. But in this discussion group about Emacs, we have members from various types of professions such as physics, molecular biology, finance, literature, etc. It is interesting how we had mostly software engineers in a mathematics discussion group but a variety of professionals in a software discussion group!

Some participants of our meetings have several years of experience with Emacs. Others are beginners. However, even those who are quite experienced with Emacs have found that they learnt many new techniques and concepts from the book. In the next few sections, I'll present some of those Emacs functions that were initially not known to some of the experienced Emacs users of our group but were found to be very useful after having learnt them from the book.

Lunar Phases

Yes, we can see the lunar phases calendar right within Emacs! I don't know if this was interesting to other members of our group, so I can only speak for myself here. As someone who has been interested in astronomy since my childhood days, I found this very exciting. Type M-x lunar-phases RET in your Emacs and a new buffer appears with an output like this:

Tuesday, March 7, 2023: Full Moon 12:39pm (UTC)
Wednesday, March 15, 2023: Last Quarter Moon 2:14am (UTC)
Tuesday, March 21, 2023: New Moon 5:27pm (UTC)
Wednesday, March 29, 2023: First Quarter Moon 2:33am (UTC)
Thursday, April 6, 2023: Full Moon 4:33am (UTC)
Thursday, April 13, 2023: Last Quarter Moon 9:17am (UTC)
Thursday, April 20, 2023: New Moon 4:16am (UTC) ** Solar Eclipse **
Thursday, April 27, 2023: First Quarter Moon 9:21pm (UTC)
Friday, May 5, 2023: Full Moon 5:32pm (UTC) ** Lunar Eclipse **
Friday, May 12, 2023: Last Quarter Moon 2:34pm (UTC)
Friday, May 19, 2023: New Moon 3:56pm (UTC)
Saturday, May 27, 2023: First Quarter Moon 3:24pm (UTC)

It also shows the upcoming eclipses! In fact, there is one coming up this month! Isn't this nice? I knew that Emacs has all sorts of fun stuff like M-x zone RET to zone out with a built-in screensaver, M-x tetris RET to play a clone of the famous puzzle game, M-: (animate-string "hello" 0) RET to display a string in a fun manner starting off as scattered pieces spread randomly across the buffer that then slide and come together to join and form the string. But to have something as obscure as lunar phases and eclipses available within the editor was a nice surprise! Thanks to the book, I now use this function often.

For people who are not familiar with Emacs notation for key binding, note that M-x lunar-phases RET means typing alt+x followed by typing lunar-phases and then pressing enter. The notation M- represents the meta modifier key which is mapped to alt on modern systems.

Returning to Mark

Most members knew that we can set a mark with C-SPC (i.e. ctrl+space) and then move around in the buffer with motion keys to highlight a region that we can cut with C-w or copy with M-w and paste it elsewhere with C-y. However, what was new to some members is the fact that we can also return to a mark just as easily. The key sequence to return to mark is C-u C-SPC. But there is a problem.

When we set a mark with C-SPC and start moving around with motion keys, the text between the mark and the current position of the cursor (known as point in Emacs) becomes a highlighted region in modern Emacs. This highlighted region can be annoying while browsing some code. So how do we use the mark as a place to return to? Barring unusual workarounds like disabling transient-mark-mode, is there a simple way? Yes, there is a simple trick. Type C-SPC C-SPC to set the mark! Typing C-SPC the first time sets the mark and activates the region. Typing it the second time deactivates the region. But Emacs remembers the mark that was set. Now continue with normal editing. Finally, type C-u C-SPC to return to mark.

The key sequences involved are pretty convenient. Typing C-SPC C-SPC involves holding down the ctrl key, then typing space twice and finally releasing the ctrl key. Similarly, typing C-u C-SPC involves holding down the ctrl key, typing u, then space and then releasing the ctrl key. Three keystrokes for each command. They become muscle memory in no time!

Working With Other Windows

In Chapter 4, The Theory of Movement, there is a section about working with other windows. There are a number of key bindings available under the prefix key C-x 4 that perform operations on another window instead of the current window. For example, C-x 4 C-f opens a file in another window. This could be a faster alternative to splitting the current window with C-x 2 or C-x 3 and then opening a file with C-x C-f. The single key sequence C-x 4 C-f takes care of splitting the current window into two if another window does not exist and opening the file there. Moreover if another window does exist, this key sequence just reuses that window to open the file there. Pretty nifty!

To see all the commands under the C-x 4 prefix key, type C-x 4 C-h. Yet another such command that I found quite convenient is C-x 4 d which opens Dired in another window. This can be useful when we need to browse a directory without hiding the current buffer.

Tab Bars

One of the many features of Emacs that most of us did not bother paying attention to earlier was the tab bar mode. Turns out it is pretty useful in managing multiple window configurations side-by-side. Each tab can be used as a workspace with a specific arrangement of windows that suits our workflow in that workspace. For example, we could arrange one tab to have three windows to display source code, a debugger and the current directory in Dired mode. Then we could have another tab with two windows beside each other, perhaps one to display some source code and another to run the terminal.

The tab management commands are very similar to window management commands. Just like C-x 2 splits a window to create a new window, C-x t 2 creates a new tab. Just like C-x 0 deletes a window, C-x t 0 deletes a tab. Similarly, C-x t o switches to the next tab. Tabs can be renamed and moved too with some more key bindings. Chapter 4 has a section called Tab Bar Mode that introduces these operations in detail.

Back to Indentation

Almost everyone knew that C-a moves the cursor to the beginning of the current line. But not many of us knew about M-m which invokes the command back-to-indentation. This command moves the cursor to the first non-whitespace character on the current line. This is another new thing some of us learnt from the book. This can be quite useful while editing code.

Exchange Point and Mark

Let us say we set a mark somewhere with C-SPC and then move around to select a region. However, then we get distracted, possibly by some typo in our buffer and we begin fixing that. At this point, the highlighted region disappears. Say after fixing that typo, we want to resume with the region selection again. What do we do now? Do we go back to set the mark and begin selecting the region again? Not really. There is an easier way. We can just type C-x C-x to exchange the point and the mark and highlight the region in between. This has the effect of reactivating the region.

Note that C-x C-x exchanges the point and the mark, so although it conveniently reactivates the region, the point jumps to where the mark was set earlier. This could be quite far from where we want the cursor to be right now. If you don't like that the cursor moves far way to the mark, just type C-x C-x once again to exchange the point and mark one more time. This has the effect of returning the cursor back to wherever it was while keeping the region activated.

Search Toggles

Say, we begin an incremental search of the literal string f.. in the current buffer with C-s f.. but then we change our mind and decide that we want to perform a regular-expression-based search using the regular expression f..? Do we cancel the incremental search and begin a new regular-expression-based search using C-M-s? That's not necessary. Instead we can toggle the currently ongoing incremental search into a regular-expression-based search using the toggle key M-s r.

Another such nice toggle is M-%. You are likely aware of the global key binding M-% used to invoke the query-replace command that performs search-and-replace operation. However, when typed during an ongoing incremental search, M-% converts the ongoing incremental search to a search-and-replace operation. This is really useful sometimes. If we are searching for a complicated string, say, C-s std::vector<int> but then we suddenly realise that we want to perform a search-and-replace operation instead with the same string, we don't really have to quit the current incremental search and start a new search-and-replace operation. Instead we can simply change the current incremental search to a search-and-replace operation by typing M-%.

Similarly, we can toggle the case-sensitivity of the search using M-s c. If we type our search string in all lowercase (e.g. C-s foo), then Emacs performs a case-sensitive search by default. I think this is a reasonable default behaviour. This is what we want most of the time. When we do want case-sensitive search for a lowercase string, we can switch from the currently ongoing case-insensitive search to a case-sensitive search with the M-s c toggle.

The moment we introduce an uppercase character in our search string (e.g. C-s Foo), Emacs switches to case-sensitive mode. Although this behaviour may appear peculiar at first, it makes sense if you think about it. If we have bothered to type an uppercase character in our search string, we probably care about the case, so Emacs switches to case-sensitive search in this case. Once again, if this is not what we want, it is trivial to change the case-sensitive search to a case-insensitive one using the M-s c toggle.

There are a number of other toggles available. Chapter 4 has a pretty long subsection on incremental search. That section discusses these toggles among many other things.

Word Search Mode

Say, we start searching for the string web_server in some code buffer with the key sequence C-s web_server. But we soon realise that the code has the words web and server written together in all kinds of notation like web->server, web::server, web-server and even web server (perhaps in inline comments). We now decide that we want to match all of them. In most other editors, we'll probably have to cancel the current search and resort to a clever regular-expression-based search. In Emacs, thanks to the toggles available in incremental search, we can type M-s w and the currently ongoing incremental search will change itself to word search mode where it now matches all these other ways those two words are written. In word search mode, Emacs searches for a sequence of words while ignoring any delimiters in between.

Occur Mode

A very useful feature that has been in Emacs for a long time and yet was unknown to many of us is the occur mode. Type M-s o foo RET and it will pull up all matches for the regular expression foo in the current buffer and display the matches in a new buffer. Now we can stay in the same buffer where we have our text and jump to the places where the matches are found using M-g M-n and M-g M-p.

It is also possible to convert an ongoing incremental search into an occur mode search using the toggle M-s o. For example, type C-s foo to start an incremental search for the string foo, then type M-s o and, lo and behold, we are now in occur mode searching for the string foo.

Imenu

Right after the section on occur mode, the book presents a section about Imenu which was also new to some of us. Type M-x imenu RET and then type TAB to invoke auto-completion and it shows a list of all interesting places in the buffer to jump too. For example, if the current buffer is a Python source code file, then the output of this command includes all functions in the Python file we can jump to. Use auto-completion to complete a function name and Imenu takes us to the place where the function is defined.

After completing the sections on occur mode and Imenu, I remember multiple members of our group mentioning that they now want to adopt occur mode and Imenu into their workflow. In fact, it may make sense to bind Imenu to a convenient key binding. The author of the book recommends binding it to M-i. However, I do use M-i sometimes to insert spaces until the next tab-stop column, so I'll suggest a different key binding that is more consistent with Emacs key binding conventions. These conventions suggest that key bindings of the form C-c letter are reserved for the users. Therefore, I suggest the key binding C-c i to invoke Imenu. Here is some Elisp code to create this key binding:

(global-set-key (kbd "C-c i") 'imenu)

Helm

Helm is a powerful filter-as-you-type framework. Most of us already knew about this package. It does too many things and has too many key bindings. We can barely scratch the surface in a small section like this. But since we discussed Imenu in the previous section, I'll mention here that Helm provides a rather nice interface to Imenu. Helm can be enabled with M-x helm-mode RET or with (helm-mode) in an Emacs initialisation file. Once enabled, C-x c i runs the helm-imenu command which provides an interactive interface with all options laid out in a vertical format. We can use the motion keys to select an option. We can then type enter to activate the selection and jump to the corresponding place in the buffer.

Helm also provides a nice interface for occur mode in the form of helm-occur command. The key sequence for it is C-x c M-s o, which one might argue is not a very convenient key binding. Nevertheless, the user interface to navigate the matches is pretty good.

Appending Kills

The key sequence C-M-w is used to ensure that if the next command happens to be a kill command, then the killed text is appended to the last stretch of text in the kill ring. This key sequence prevents the next kill from creating a new entry in the kill ring.

To understand what this command does we must first understand that after a kill command adds some new text to the kill ring, subsequent consecutive kills append to the same stretch of text in the kill ring, i.e. consecutive kills form a single large stretch of text in the kill ring. This can be tested by performing consecutive kills and then pasting with C-y. For example, M-d M-d M-d kills 3 words and creates a single stretch of text consisting of those 3 words. The consecutive kills keep adding to the same kill entry in the kill ring. If we type C-y now, it would yank the newest entry consisting of those 3 words from the kill ring and paste it into the buffer.

However, the moment a non-kill command is used, it seals the current entry in the kill ring. Any subsequent kill command creates a new entry in the kill ring. For example, M-d M-d M-d C-p M-d M-d C-p C-p C-y kills 3 words at first but then it moves to the previous line sealing that kill entry consisting of 3 words. Then it kills 2 more words and adds them to a new entry in the kill ring. Therefore, the final yank command pastes only those 2 words from the kill ring.

This can be a problem if we want to kill text from various parts of the buffer and yet create a single entry in the kill ring. That's when C-M-w comes useful. For example, M-d M-d M-d C-p C-M-w M-d M-d C-p C-p C-y kills 3 words and creates a single entry in the kill ring consisting of those 3 words. Then it moves one line up and kills 2 more words but this time it appends those 2 words to the existing entry in the kill ring. Finally, it moves two lines up and pastes the entire kill entry consisting of 5 words into the buffer.

Yank-Pop

Although most beginners and experienced users of Emacs know that killed text goes into the kill ring and we can yank the last kill from kill ring and paste it into the buffer using C-y, for many that's where the usage of kill ring stops. The kill ring, however, has much more utility than that. It is a ring, after all! We can keep killing text as we edit text and all killed text gets added to the kill ring. Now C-y always yanks the newest kill in the kill ring and pastes it in the buffer? Can we recall an older kill? Yes, using the M-y key sequence. This key sequence invokes the yank-pop command that replaces a just-yanked stretch of killed text with an older kill. It takes a little bit of getting used to but once this becomes muscle memory, the kill ring becomes a very powerful tool. We can keep dumping text to the kill ring and keep recalling text from it while performing our editing activities. The following exercise shows how C-y and M-y can be used together.

Open a new file, say, with C-x C-f foo.txt RET and type these five words in a single line: foo bar baz qux quux.
Then type C-a M-d C-g M-d C-g M-d. At this point three stretches of text have been inserted into the kill ring. The C-g between every M-d is there only to avoid appending kills to the existing stretch of text in the kill ring. This ensures that we have three separate kills in the kill ring.
Now type C-y. The last kill, i.e. baz is now pasted into the buffer.
Now without typing any other key sequence, type M-y. The earlier pasted text baz is now replaced with an older stretch of text from the kill ring. Thus baz is replaced with bar.
Now once again type M-y. The earlier pasted text bar is now replaced with a further older stretch of text from the kill ring. Thus bar is replaced with foo.

Join Us

That was an account of our Mastering Emacs book club discussions so far and a few interesting things we learnt. We have only recently begun reading Chapter 5 that introduces several editing and text manipulation commands. This chapter is 75 pages long and it could take a month or two to complete this chapter. There are two more chapters after that which are shorter in their lengths. They discuss some practical aspects of Emacs along with a discussion on some popular packages. We still have a long way to go before we can complete this book.

If all of this sounds like fun, you are very welcome to join our meetings. Just head over to cc/mastering-emacs/ and there you'll find everything you need to know in order to be a part of our book discussion group and join our meetings. Looking forward to seeing you in our next meeting!

Update on 30 Dec 2023: Our discussions of this book concluded on 30 Dec 2023 after 72 meetings. See the post From Fill Prefix to Tramp for more highlights from these meetings.

Read on website | #emacs | #meetup | #technology

Notes on Mastering Emacs: Chapter 4: The Theory of Movement

2023-03-25T00:00:00Z

The following notes were taken while discussing Chapter 4 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Basics
Major Mode Load Order
- File-Local Variables
Occur Mode

Basics

The following complete key sequences illustrate a few basic commands:

C-x C-f foo.txt RET: Edit file named foo.txt.
C-x C-s: Save current buffer to file.
C-x b *scratch* RET: Switch to the buffer named *scratch*.
C-x k *scratch* RET: Kill the buffer named *scratch*.
C-x k RET: Kill current buffer. In fact, like the previous key sequence above, typing C-x k first prompts for the buffer name. However, the current buffer name is selected as the default value already. As a result, typing RET kills the current buffer.
C-x C-b: List buffers.
C-x C-c: Exit Emacs. This command offers to save all unsaved buffers before exiting Emacs.
ESC ESC ESC: This command exits the current context. What that means depends very much on the context. It performs exactly one of the following actions: If there is an active region, then it is deactivated; if a minibuffer is open, it gets rid of it; if recursive edit is in progress, it quits one level of recursive editing; if multiple windows are open, it deletes other windows so that the current window becomes the only window in the frame. The aforementioned conditions are tested one by one and as soon as one of the conditions is met, the corresponding action is executed and the other conditions are skipped.
C-/: Undo changes.
F10: Activate the menu bar.

In my experience, I have found that ESC ESC ESC is most useful when a stray minibuffer is open but the cursor is on some other buffer instead of the minibuffer and I need to close the minibuffer. Here are some steps that demonstrate this usage:

Type M-x white and pause. We now have a partially typed command in the minibuffer.
Now pretend that we get distracted by some imperfections in the text buffer that was open earlier and we want to fix those first. Type C-x o to move away from the minibuffer and go back to the text buffer to perform some editing tasks.

In this step, we could have typed C-g to quit the minibuffer first but we did not do that. We pretended to get distracted by the text buffer and went straight to it from the minibuffer by typing C-x o. At this point, the cursor is in the text buffer and the minibuffer remains open at the bottom. The open minibuffer can be distracting while performing the text editing tasks. Typing C-g now will not get rid of the minibuffer because the cursor is no longer in the minibuffer.
Now one way to close the open minibuffer could be to type C-x o to go back to the minibuffer window and type C-g. However, there is a more direct way to do this as explained in the next point.
Type ESC ESC ESC to get rid of the minibuffer at the bottom. This works even when the cursor is not in the minibuffer but is in the text buffer instead.

Major Mode Load Order

The chapter mentions the following order for detecting major mode:

File-local variables
Program loader directives
Magic mode detection
Automatic mode detection

Let us start from the bottom of the list and share some experimental results that illustrate how the major mode detection works.

File-Local Variables

Create a text file named foo.txt with the following content:
```
#include <iostream>

int main() {
  std::cout << "hello, world\n";
  return 0;
}
```
Then open this file in Emacs (say, with C-x C-f foo.txt RET). Emacs sets the major mode to Text (i.e. text-mode).

Type M-: major-mode RET to confirm that indeed the value of major-mode is text-mode. This happens due to automatic mode detection which determines the major mode based on the file name. In this case it sees that the file name ends with .txt and enables text-mode. We will discuss automatic mode detection further in the section Automatic Mode Detection.
Now edit the previous file to add file-local variables in the header as follows:
```
// -*- mode: c++; c-basic-offset: 4 -*-

#include <iostream>

int main() {
  std::cout << "hello, world\n";
  return 0;
}
```
Now reload the buffer. You could simply kill the buffer with C-x k and reopen the file with C-x C-f foo.txt RET or alternatively, reload the buffer with M-x revert-buffer RET yes RET.

After reloading the buffer, you should see that the C++ mode (i.e. c++-mode) is enabled. As a result, C++ syntax highlighting should be visible. Further C-x h TAB should reformat the code to use 4 spaces for each level of indentation.
File-local variables may be specified in the footer too as shown below:
```
#include <iostream>

int main() {
    std::cout << "hello, world\n";
    return 0;
}

// Local Variables:
// mode: c++
// c-basic-offset: 6
// End:
```
Now reloading the buffer should show that c++-mode is enabled and typing C-x h TAB should reformat the code to use 6 spaces for each level of indentation.
What happens if the file-local variables in the header and footer contradict each other? To test this out, edit the buffer to have the following content:
```
// -*- mode: c++; c-basic-offset: 4 -*-

#include <iostream>

int main() {
      std::cout << "hello, world\n";
      return 0;
}

// Local Variables:
// mode: python
// c-basic-offset: 6
// End:
```
Reloading the buffer should show that c++-mode is active. Typing C-x h TAB should reformat the code to use 6 spaces for each level of indentation. Therefore the mode specified in the header remains effective. For other variables, the ones specified in the footer have precedence.

Of course, the example above is intended for curiosity and exploration. In practical use, however, it is best to avoid assigning conflicting values to file-local variables in both the header and footer. Such inconsistencies can lead to confusion and make the effect of the variables difficult to understand.

Occur Mode

TODO: More notes coming up here soon!

Read on website | #emacs | #technology | #book | #meetup

Notes on Mastering Emacs: Chapter 3: First Steps

2023-01-07T00:00:00Z

The following notes were taken while discussing Chapter 3 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Starting Emacs
Emacs Client-Server
The Emacs Interface
Keys
Terminal Limitations
Definitions for Key Sequences
Key Examples
C-g: Universal Bail Me Out
Caps Lock as Control
M-x: Execute Extended Command
Interactive Commands
M-X: Execute Extended Command for Buffer
Universal Arguments
Discovering and Remembering Keys
True Colour
The Customise Interface
Customise Commands
Evaluating Elisp Code
Info
Apropos
Describe
Links

Starting Emacs

Here are some frequently used commands to start Emacs:

emacs -nw: Tell Emacs not to create a graphical frame, i.e. run Emacs within a terminal instead.
emacs -q: Do not load an init file, i.e. do not load ~/.emacs, ~/.emacs.d/init.el, etc.
emacs -Q: Do not load init file, site-wide startup file or X resources.

Emacs Client-Server

The key sequence M-x server-start RET turns the current instance of Emacs into a server. If we kill this instance of Emacs, it kills the server too.

Another way to start an Emacs server is to enter the command emacs --daemon. This too internally invokes server-start to start an Emacs server but this command ensures that Emacs runs in background mode. Therefore, killing any particular instance of Emacs window does not end up killing the Emacs server.

To open files, say foo.txt and bar.txt in an already running Emacs server, enter the command emacsclient foo.txt bar.txt. This command blocks the terminal and waits for us to finish editing the files. While editing the files, we need to type C-x # to tell Emacs that we are done editing the current file. Emacs then switches to the next file we are editing. When we are done editing all the files opened, emacsclient quits and returns control to the terminal.

The book does not mention how to stop an Emacs daemon. One of the several ways to stop an Emacs daemon is the command emacsclient -e '(kill-emacs)'.

Here are some commands to run emacsclient:

emacsclient -c: Create a frame. A graphical frame is created if graphics is available, otherwise a terminal frame is created.
emacsclient -nw: Create a terminal frame.
emacsclient -n: Client returns immediately without waiting for us to finish editing the file.

The Emacs Interface

The book mentions that many Emacs users disable the menu bar, tool bar and the splash screen. The following Elisp code shows how these UI elements can be disabled:

(menu-bar-mode 0)
(when (display-graphic-p)
  (tool-bar-mode 0)
  (scroll-bar-mode 0))
(setq inhibit-startup-screen t)

After saving the code in the initialisation file (such as ~/.emacs, ~/.emacs.d/init.el, etc.) and restarting Emacs, these UI elements disappear.

It is worth mentioning here that disabling the menu bar may not be a good idea, especially, for beginners to Emacs. The menu bar contains a lot of helpful shortcuts that could be useful to beginners. Further, the menu bar often displays certain menus that are specific to the current buffer. Therefore, it may be a good idea to leave the menu bar enabled.

Regardless of whether the menu bar is enabled or disabled, the menu bar can accessed easily by typing F10.

Keys

The book mentions the following notation for modifier keys:

C-: Control
M-: Meta
S-: Shift
s-: Super
H-: Hyper
A-: Alt

Although not mentioned in the book, here is a quick way to test out all of these modifier keys:

(global-set-key (kbd "C-j") (lambda () (interactive) (message "You typed C-j")))
(global-set-key (kbd "M-j") (lambda () (interactive) (message "You typed M-j")))
(global-set-key (kbd "C-S-j") (lambda () (interactive) (message "You typed C-S-j")))
(global-set-key (kbd "s-j") (lambda () (interactive) (message "You typed s-j")))
(global-set-key (kbd "H-j") (lambda () (interactive) (message "You typed H-j")))
(global-set-key (kbd "A-j") (lambda () (interactive) (message "You typed A-j")))

Go to some buffer, say, the scratch buffer with C-x b *scratch* RET, then copy the above code to it, then place the cursor at the end of each line of code and type C-x e to evaluate each line.

Then open a new buffer, say with C-x b foo RET and type ctrl+j, alt+j and ctrl+shift+j to test the first three key bindings.

The fourth key-binding s-j can usually be invoked by typing command+j or win+j depending on the type of keyboard you have.

The H- and A- modifier keys are generally not mapped to any actual key in modern systems. However, it is possible to invoke the H-j and A-j key bindings with the key sequences C-x @ h j and C-x @ a j respectively, i.e. ctrl+x @ h j and ctrl+x @ a j respectively.

In fact, similarly, s-j too can be invoked with C-x @ s j, i.e. ctrl+x @ s j but that is rarely necessary because s- is often bound to a GUI key like the command key on Apple keyboards and the win key on Windows keyboards.

A practical example of a real and useful s- key binding from vanilla Emacs is s-u to invoke the revert-buffer command that reloads a file from the disk.

Terminal Limitations

The book mentions that there are some key bindings that we cannot use if we are running Emacs in the terminal. That is because a terminal supports a very limited set of key bindings. An example is C-/ that invokes the undo command in GUI Emacs. The terminal does not recognise that key sequence. However, the undo command is also bound to C-_ and C-x u, so one of these key sequences can be used to undo changes in terminal Emacs.

Definitions for Key Sequences

The book provides a few definitions of key sequences that can be summarised as follows:

Key sequence (or just key): A sequence of keyboard or mouse actions, e.g. C-d, C-M-d, C-x C-f, C-x 8 P, C-x, C-x 8, etc.
Complete key: A key sequence that invokes a command, e.g. C-d, C-M-d, C-x C-f, C-x 8 P, etc.
Prefix key: A key sequence that is not a complete key, e.g. C-x, C-x 8, etc.

Key Examples

The section Keys presents the following examples of key sequences:

C-d: Calls delete-char which deletes the following character.
C-M-d: Calls down-list which moves forward down one level of parentheses.
C-x C-f: Calls find-file which is used for editing a file.
C-x 8 P: Insersts pilcrow, i.e. the symbol ¶, also known as the paragraph symbol. Here both C-x and 8 are prefix keys. There are many more key bindings available under the C-x 8 prefix. For example, C-x 8 C inserts the copyright symbol, i.e. the symbol ©. A set of keys like this that belong to a particular prefix key is called a key map.
C-M-%: Calls query-replace-regexp to replace text with regula-expression based matching. Since the key % is normally typed as shift+5, this key sequence involves holding down 4 keys together, i.e. ctrl+alt+shift+5. Annoyingly, this happens to be an example of a key sequence that does not work in terminal Emacs due to terminal limitations.
TAB: Calls indent-for-tab-command that either indents the current line or region or inserts a tab, depending on the context.
<f1>, <f2>, ..., <f10>: Functions keys that invoke various functions. Some of these are prefix keys while others are complete key sequence. For example, <f1> is a prefix key that is available as an alternative to the C-h prefix key. The key sequence C-h m shows help for the current major and minor modes and so does <f1> m. However, <f10> is a complete key sequence that calls the command menu-bar-open that shows the menu bar.

C-g: Universal Bail Me Out

The key sequence C-g is used to cancel a partially completed command. For example, normally, the key sequence M-x whitespace-mode RET toggles the visibility of whitespace in the current buffer. However, let us say, we type the partial key sequence M-x white and then we change our mind about it and want to cancel entering the command any further, we can simply type C-g to run keyboard-quit that signals a quit condition and cancels the input.

Similarly, say, we type C-x and then we change our mind about it and want to cancel this partially entered key sequence, we can simply type C-g. The following message appears in the echo area but this is by design:

C-x C-g is undefined

Don't let that message make you feel that you did something wrong by entering an undefined key sequence. Key sequences ending with C-g have been intentionally left undefined, so that it can be used reliably as the universal bail me out key sequence.

Now of course, nothing stops us from binding C-x C-g to a command of our choice. For example, the following Elisp code binds it to a command that prints a message:

(global-set-key (kbd "C-x C-g") (lambda () (interactive) (message "You typed C-x C-g")))

However, a key binding like the above one is a very bad idea because such a key binding flies against Emacs conventions. If we were to create a key binding like the above one, we can longer rely on C-g to be our universal bail out command. Key sequences ending with C-g are best left undefined.

Caps Lock as Control

The book recommends configuring our operating system to make the caps lock key behave like ctrl. Many people do find this type of remapping very convenient. Many discussions can be found online where people have claimed that this remapping has helped with overcoming repetitive strain injury (RSI). However, the article Moving the Ctrl Key on Emacs Wiki claims that for some people this remapping causes RSI.

I use the original ctrl keys as they come with the keyboard. Most keyboards have two ctrl keys on either side of the keyboard which I find very convenient. I touch type while editing text, so the two ctrl keys on either side of the keyboard turn out to be really useful. For example, when I need to type ctrl+a, I can hold down ctrl with the little finger of the right hand and type a with the little finger of the left hand. Similarly, if I need to type ctrl+p, I can hold down ctrl with the little finger of the left hand and type p with the little finger of the right hand. Using the original ctrl keys offers this advantage of distributing the usage of the ctrl to both hands. However, some Apple keyboards provide only a single ctrl key on the left hand side which can be quite annoying to touch typists. In such keyboards, remapping the caps lock key to behave like ctrl key can indeed be more convenient.

M-x: Execute Extended Command

The key sequence M-x is pronounced mex, M x or meta x. It invokes the command execute-extended-command that brings up a minibuffer to read a command name and execute it.

When M-x key sequences are presented in written form, often they may be written in a precise manner that includes the M-x key sequence, then the command name and finally the RET key in the end, e.g. M-x lunar-phases RET. But sometimes they may also be written without the RET key, e.g. M-x lunar-phases. The RET key is automatically implied in the latter form.

Interactive Commands

The following function written in Elisp is interactive:

(defun hello ()
  (interactive)
  (message "hello"))

However, the following function is not interactive:

(defun hola ()
  (message "hola"))

The (interactive) expression declares a function as interactive which allows it to be called interactively using M-x.

To see the difference between the two functions, copy both functions to some buffer, say, the scratch buffer and then put the cursor after the closing parentheses of each function and type C-x C-e to evaluate the defun expressions thus defining the functions. After doing so, M-x hello RET executes the hello function and produces the "hello" message in the echo area. However, typing M-x hola RET produces no match for a function named hola.

M-X: Execute Extended Command for Buffer

Emacs 28 introduces M-X that runs the command execute-extended-command-for-buffer. The book mentions this key binding as M-S-x which is in fact M-X. In these notes, we'll write M-X for consistency with how this key sequence is actually defined and represented in Emacs (for example, note that C-h k M-S-x shows the key as M-X).

To see how this function works, first open a file, say, foo.el with C-x C-f foo.el RET, then type M-x e RET. A large list of function names beginning with the letter "e" as choices appears. Now type M-X e RET. A much smaller list of choices that are relevant for the current Elisp buffer appears.

Universal Arguments

Universal arguments are also called prefix arguments. The list below presents some examples of complete key sequences where we apply various prefix arguments to the key sequence C-n.

C-u C-n: Move cursor down 4 lines.
C-u C-u C-n: Move cursor down 16 lines.
C-u C-u C-u C-n: Move cursor down 64 lines.
C-u 5 C-n: Move cursor down 5 lines.
C-u 15 C-n: Move cursor down 15 lines.
C-5 C-n: Move cursor down 5 lines.
C-1 C-5 C-n: Move cursor down 15 lines.
C-- C-n: Move cursor up 1 line.
C-- C-5 C-n: Move cursor up 5 lines.
C-- C-1 C-5 C-n: Move cursor up 15 lines.

Note that negative arguments (the last three examples above) reverses the direction of operation. The digit arguments of the form C-<digit> can also be entered using M-<digit> or C-M-<digit>. Here are some examples:

M-5 C-n: Move cursor down 5 lines.
C-M-5 C-n: Move cursor down 5 lines.
M-1 M-5 C-n: Move cursor down 15 lines.
C-M-1 C-M-5 C-n: Move cursor down 15 lines.
M-- C-n: Move cursor up 1 line.
M-- M-5 C-n: Move cursor up 5 lines.
M-- M-1 M-5 C-n: Move cursor up 15 lines.
C-M-- C-n: Move cursor up 1 line.
C-M-- C-M-5 C-n: Move cursor up 5 lines.
C-M-- C-M-1 C-M-5 C-n: Move cursor up 15 lines.

While entering multiple digits in a digit argument, we can also mix and match modifier keys as shown below, however doing so would be pointless and unwieldy:

C-1 M-5 C-n: Move cursor down 15 lines.
C-1 C-M-5 C-n: Move cursor down 15 lines.

The reason why Emacs supports entering digit arguments or negative arguments with all three modifier combinations C-, M-, C-M-, is so that we can choose a modifier that is convenient. Usually, this would be the modifier we anticipate that we'll use next for the command we are prefixing with the digit or negative argument. For example, if we are going to move forward by 5 words, then M-5 M-f is going to be more convenient than C-5 M-f or C-M-5 M-f. To type M-5 M-f, we can simply press and hold down alt, then type 5 followed by f and finally release alt. Similarly, if we are going to move forward by 5 expressions, then C-M-5 C-M-f is going to be more convenient than using any other modifier for the digit argument.

Choosing the modifier combination for a universal argument such that it matches the modifier combination for the key sequence coming up next helps maintain good tempo while typing the key sequences. The book often puts emphasis on the subject of tempo in various chapters.

While discussing tempo, the book presents the example of M-- M-d which can be used to kill the previous word. Note that M-d kills one word forward, so M-- M-d reverses the direction of the operation and kills one word backward. The book notes that M-- M-d maintains tempo while C-- M-d which does exactly the same thing disrupts tempo.

Another example that the section Univeral Arguments briefly alludes to but does not provide a concrete example of is changing the case of a word that we just typed. Here are some concrete examples for it:

M-l: Convert the text from the point to the end of the current or next word to lower case.
M-u: Convert the text from the point to the end of the current or next word to upper case.
M-c: Capitalise the text from the point to the end of the current or next word. Capitalisation involves changing only the first letter to upper case.
M- M-l: Convert the text from the point to the beginning of the current or previous word to upper case.
M- M-u: Convert the text from the point to the beginning of the current or previous word to upper case.
M- M-c: Capitalise the text from the point to the beginning of the current or previous word.

Discovering and Remembering Keys

To get help for a prefix key, type the prefix key followed by C-h. Here are some examples:

C-x C-h
C-x 8 C-h
C-x 8 " C-h
C-x 8 ' C-h
C-x 8 * C-h

Getting help for a prefix key in this manner shows an automatically generated list of all keys that belong to the key map associated with the prefix key. Note that a key map for a prefix key may itself contain more prefix keys. For example, in the above examples, we see that the key map for C-x 8 contains prefix keys C-x 8 ", C-x 8 ', etc. Such nested prefix keys are clearly marked in the output as "Prefix Command".

True Colour

There is a note with the title Supported colors in the section The Customize Interface that introduces the following commands:

M-x list-color-display RET: Display names of defined colours and show what they look like.
M-x info-apropos RET Colors on a TTY RET: Find an info page on the subject of using colours on a TTY.

Since Emacs 28, we can easily enable 24-bit colour in terminal Emacs by setting the environment variable COLORTERM=truecolor. To quickly test it out, first enter the following command in the terminal:

COLORTERM=truecolor emacs -nw

Then type the following in Emacs:

M-x list-color-display RET

The Customise Interface

The customise interface allows us to customise two things: faces and options. Here are some steps to get started with the customise interface:

Type C-x 2 to split the current window into two.
Type C-x C-f foo.el RET to open a new Elisp file.
Type some code into the file such that it contains at least one string, for example:
```
(message "hello")
```
Type C-x o to go to the other window.
Type M-x customize RET.
Move the cursor to the editable text area. This can be done with motion key sequences or with the mouse.
Then type font-lock-string-face and press enter. The customisation interface for the chosen face now appears in the buffer. The message next to the "State" button shows "STANDARD." which indicates that the face is set to its default value.
Then move the cursor to the editable text area next to the "Foreground:" label, type blue. At this point, a preview of blue text is shown next to the "Choose" button. However, the string in the Elisp buffer still appears in its previous colour. The message next to the "State" button shows "EDITED, shown value does not take effect until you set or save it."
Now click the "Apply" button to apply the new string colour. Alternatively, move the cursor over to "Apply" and press enter. As soon as this button is clicked, the string in the Elisp buffer appears blue. The message next to the "State" button says "Set for current session only."
Now click the "Revert..." button and then click "Revert This Session's Customizations" to revert the customisation.
Now change the string colour to blue again and click "Apply and Save". The message next to the "State" button now changes to "SAVED and set." Then check Emacs initialisation file (such as ~/.emacs, ~/.emacs.d/init.el, etc.) and there should be a custom-set-faces call added to the initialisation file to set the string colour to blue.
To erase the customisation from the initialisation file, click "Revert ..." and then click "Erase Customizations". At this point, the customisation is no longer present in the initialisation file. However, the customise interface shows a confusing message next to the "State" button, "EDITED, shown value does not take effect until you set or save it." Further, the face is set to "-- Empty face --" in the customisation interface. To add to the confusion, the "Apply and Save" button is enabled! Beware though! If "Apply and Save" is clicked now, it will set nil as the value for the face which will cause the face to appear black or white (not the default colour).
To get rid of the confusing state of the customisation buffer discussed in the previous point and restore the customisation buffer to a sane state, click "Revert ..." and then click either "Undo Edits in Customization Buffer" or click "Revert This Session's Customizations". Doing so would end up showing the default colour of the face in the customise interface. The message next to the "State" button shows "STANDARD." again.

Customise Commands

Here are some examples of complete key sequences that invoke various customise commands:

M-x customize RET: Display customise interface. We can reach a specific item to be customised by navigating the groups presented and the subgroups within them. For example, to customise font-lock-string-face, we can naviage to Faces > Font Lock > Font Lock Faces > Font Lock String Face.
M-x customize-browse RET: Create a tree browser for the customise hierarchy.
M-x customize-option RET global-display-line-numbers RET: Customise a specific option.
M-x customize-face RET font-lock-string-face RET: Customise a specific face.
M-x customize-group RET faces RET: Customise a specific group.
M-x customize-group RET font-lock-faces RET: Another example similar to the previous one. This one customises the Font Lock Faces subgroup within the Faces group.
M-x customize-mode RET: Customise the major mode of the current buffer.
M-x customize-mode RET python-mode RET: Customise a specific mode. After the first RET, a prompt to enter the mode appears only if the current major mode has no known customise group.
C-u M-x customize-mode RET python-mode RET: Yet another way to customise a specific mode. Using the prefix argument ensures that we are always prompted for the mode name. This can be useful when we want to customise a minor mode or a major mode that is different from the current major mode.
M-x customize-themes RET: Display a selectable list of custom themes.
M-x customize-customized RET: Customise all customisations set in the session but not saved. This is very useful to see all unsaved customisations together at one place.
M-x customize-saved RET: Customise all saved customisations. This is very useful to see all saved customisations together at one place.
M-x customize-changed RET: Customise all settings whose meanings have changed since the previous major Emacs release.
M-x customize-changed RET 26.1 RET: Customise all settings whose meanings have changed since a particular previous major Emacs release.

One thing worth keeping in mind is that when we apply, save or revert customisations, only the customisations shown in the current buffer are applied, saved or reverted. Here is an experiment that demonstrates what this means:

Type C-x 2 followed by C-x C-f foo.el RET to split the current window into two and load an Elisp file in one window. Then insert the following code:
```
(defun foo () (message "hello")) ;; Demo
```
Type M-x customize RET, search for font-lock-keyword-face, set its colour to red and click "Apply".
Then search for font-lock-function-name-face, set its colour to green and click "Apply". At this point the macro name defun is coloured red and the function name foo is coloured green in the Elisp buffer.
Now click "Revert ..." and then click "Revert This Session's Customizations". Only the colour of the function name reverts to the default colour. The colour of the macro name remains red in the Elisp buffer. This confirms that the revert operation takes only the customisations in the buffer into account.
Search for font-lock-string-face, set its colour to blue and click "Apply and Save".
Search for font-lock-comment-face, set its colour to magenta and click "Apply and Save". Inspecting the Emacs initialisation file shows that only the customisations for the string and comment faces have been saved to the file. This confirms that only the customisations shown in the current buffer are saved.
Type M-x customize-customized RET. A new buffer appears and shows the customisation for keyword because it was customised and applied but not saved.
Type M-x customize-saved RET. A new buffer appears and shows the customisations for string and comment because they were customised and saved.
Now search for font-lock-comment-face so that the customise buffer contains only the customisation entry for comment. Click "Revert ..." and then click "Erase Customizations". Inspecting the Emacs initialisation file shows that the customisation for comment is removed but the customisation for string is still intact. This confirms that only the customisation shown in the current buffer is erased.

Evaluating Elisp Code

The chapter introduces two commands:

M-x eval-buffer RET: Evaluate the entire buffer.
M-x eval-region RET: Evaluates only the marked region.

Here are some additional key sequences that I find very useful:

C-x C-e: Evaluate the expression before the cursor. This key binding is available in global-map, so this key sequence works in any buffer.
C-M-x: Evaluate the top-level expression on the cursor. This key binding is not available in the global map. Therefore this key sequence works only in those major modes that support it. For example, this key sequence works in Elisp buffers because this key binding is available in emacs-lisp-mode-map.

Info

Info, also known as the Info Reader, is the documentation browser of Emacs. Type M-x info RET or C-h i to enter Info.

The following points explain the navigation commands of Info:

Type [ or ] to go to the previous or next node respectively. Typing ] repeatedly is a nice way to go from one node to the next one without skipping any content. This provides an experience that very close to reading a book page by page although SPC (explained below) might be an even better way.
Type l or r to go back or forward in history respectively.
Type n or p to go to the previous or next sibling node. This is a nice way to go from one chapter to next while skipping all the child nodes or to go from one section to the next sibling section while skipping all child subsections.
Type u to go up one level to the parent node.
Type SPC to scroll one screen forward at a time. Type DEL or S-SPC to scroll one screen backward at a time. When a node boundary is reached, these commands automatically load the next or previous node. Thus typing SPC over and over again is a nice way to read all the text in a node and then go to the next node. This provides an experience that is closest to reading a book paragraph by paragraph and page by page.
Type TAB to go to the next cross-reference or menu item, i.e. anything that looks like a link. Type S-TAB or C-M-i to go back to the previous cross-reference or menu item.
Type RET to follow a node reference near point. This behaves like visiting a link.
Type m to prompt for a menu item to go to. The menu item near the point is automatically selected as the default answer to the prompt.
Type q to quit Info. The Info buffer is not killed. Therefore typing C-h i will switch to the Info buffer and we can read the last node we visited again.

To summarise, the commands [ and ] are great for browsing every node. The command SPC and DEL are good for reading everything paragraph by paragraph and page by page.

There are several ways to reach a specific node of a specific manual. The following points describe some of them:

When we open the Info reader with C-h i for the first time, immediately typing m is a convenient way to enter a particular menu item for a manual and jump to that manual, e.g. type C-h i m org RET to go to the Org manual.
Since Emacs 28, we can also use C-h R to open a specific manual from any buffer, i.e. we do not need to type C-h i first to enter Info. For example, type C-h R org RET to enter the Org manual.
To look up the documentation for a command, type C-h F, e.g. C-h F find-file RET.
Although not mentioned in the book, a popular way to point others to a specific node in the manual is to provide an Elisp expression to visit a node. For example, to read the node named Internal Links in the org manual, evaluate the expression (info "(org)Internal Links").
An alternative to the Elisp expression in the previous point could be typing C-h R org RET m Hyperlinks RET m Internal Links RET or even C-h R org RET m Internal Links RET. However, the Elisp expression in the previous point is a convenient way to share a pointer to a specific node in a specific manual with other Emacs users.

Apropos

The following key sequences demonstrate how to use the apropos system:

C-h a ^whitespace RET: Find all commands that match the regular expression pattern ^whitespace.
M-x apropos-command RET ^whitespace RET: Same as above.
M-x apropos RET ^whitespace RET: Show all symbols that match the given regular expression pattern. The matches include symbols defined as functions, variables and faces.
C-h d cleanup-region RET: Show symbols whose documentation strings match the given regular expression pattern. Note that this command shows symbols like whitespace-cleanup and whitespace-report-region in the results because their documentation strings contain the string "cleanup-region". However, it does not show the symbol whitespace-cleanup-region because its documentation string does not contain this string.
M-x apropos-documentation RET cleanup-region RET: Same as above.
M-x apropos-library RET whitespace RET: Show all variables and functions defined by the library whitespace.
M-x apropos-user-option RET ^whitespace RET: Show user options that match the given pattern. This includes variables that can be customised in the customise interface.
M-x apropos-value RET ^[^ ]*text RET: Show all symbols which have values whose printed representation matches the given pattern.

By default, the apropos commands show the results in alphabetical order. To sort them by scores, evaluate the following Elisp expression:

(setq apropos-sort-by-scores t)

To sort the matches by score as well as show the scores within parentheses next to the matches, evaluate the following Elisp expression:

(setq apropos-sort-by-scores 'verbose)

Note that sorting by score does not work for C-h d (apropos-documentation).

Describe

The following key sequences demonstrate the describe system:

C-h m: Display documentation of current major mode and minor modes along with key bindings introduced by the modes. Mode-specific commands that are not bound to any key are not shown.
C-h x calc RET: Display documentation of the calc command.
C-h f calc RET: Same as above. However, note that while C-h f can display documentation of any function, C-h x on the other hand can display documentation of only those functions that are also commands. For example, C-h f string-join RET works fine but C-h x string-join RET complains about no match.
C-h v tab-width RET: Display documentation of the variable tab-width.
C-h k C-x C-f: Display documentation of the command bound to the key sequence C-x C-f.

Links

The following list includes some links that were discussed during the book discussion group meetings:

Read on website | #emacs | #technology | #book | #meetup

Notes on Mastering Emacs: Chapter 2: The Way of Emacs

2022-12-18T00:00:00Z

The following notes were taken while discussing Chapter 2 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Magpie's Nest of Shiny Things
Cornerstone of Emacs
Uptime
RPN Calculator
Point and Mark
Returning to Mark
Font Locking
Change Major Mode
Links

Magpie's Nest of Shiny Things

The following commands mentioned in the book invoke some interesting functions:

M-x zone RET: Zone out, completely; a built-in screensaver.
M-x dunnet RET: Dungeon text adventure game.
M-x tetris RET: Tetris clone.
M-x lunar-phases RET: Lunar phases calendar.
M-x doctor RET: ELIZA chatterbot.

Cornerstone of Emacs

The chapter mentions the Elisp interpreter as the cornerstone of Emacs. In fact, Emacs is an editor program that runs in the Elisp interpreter. When we start Emacs, what really starts first is an Elisp interpreter. It executes the Elisp code of an editor program. When that program executes, we get Emacs, the editor!

This behaviour is quite the opposite of how Vim and other editors behave. For example, when we start Vim or Visual Studio Code, first the editor runs. The editor then provides an embedded programming language that we can use to write plugins and extend the editor. For example, Vim offers VimScript to let us write code using VimScript and extend the functionality of Vim. However, when we start Emacs, first the Elisp interpreter runs and this interpreter executes a program that results in Emacs, the editor.

Uptime

The key sequence M-x emacs-uptime RET shows how long the current instance of Emacs has been running. Many users have month-long uptimes.

RPN Calculator

To try out the calculator, type C-x * c or M-x calc RET to start the calculator. Then to calculate something like, say, 1 + (2 * 3), type 1 RET 2 RET 3 RET * +.

Point and Mark

To try out point and mark quickly, first open a buffer with some text in it. For example, open a file, say, foo.txt using the key sequence C-x C-f foo.txt RET. Make sure there are a few lines of text in the buffer. If it is a new buffer, type some lines of text into it. Then move the point (cursor) to any arbitrary place within the text. The motion keys like C-p, C-n, C-b, C-f, etc. or simply <up>, <down>, <left>, <right>, etc. may be used to do this.

Then type C-SPC (i.e. ctrl+space) to set the mark at the current position of the point. Then move the point around. Emacs should now be highlighting the region between the mark (set earlier) and the point. Then type M-w (i.e. alt+w) to copy the text in the selected region or type C-w to cut the text instead. Finally, move the point around again and type C-y to paste the copied/cut text.

Returning to Mark

Marks can also be used to remember a position in the buffer to which we may wish to return to later. To try it out, first move the point to some place in the buffer where a mark needs to be set. Then type C-SPC C-SPC to set the mark without activating it (activating a mark causes Emacs to highlight the region as explained in the previous section). Then move the point to some other place in the buffer. Finally, type C-u C-SPC to return to the marked position.

Font Locking

Font locking (syntax highlighting) in Emacs is made up of faces of properties (colour, font, size, etc.). Type C-u C-x = to describe the current character (the character the cursor is on). The output displays the face details as well. Alternatively, type M-x describe-face RET to describe the face of the character the cursor is on.

Change Major Mode

Emacs almost always sets the appropriate major mode by inspecting the filename or the content of the buffer. However, sometimes it can be useful to set the major mode manually. For example, consider a file named foo.html which is open in Emacs and has the following text:

<h1>Euler's Identity</h1>
<p>
  In mathematics, <em>Euler's identity</em> is the
  equality

  \[
    e^{i \pi} + 1 = 0.
  \]

  Euler's identity is a special case of Euler's formula from complex
  analysis, which states that for any real number \( x, \)

  \[
    e^{ix} = \cos x + i \sin x.
  \]
</p>

This is an HTML file with some LaTeX snippets that would perhaps be rendered using a LaTeX rendering tool such as MathJax or KaTeX. By default, when this file is opened in Emacs, the major mode is set to HTML+. However, if one wants to focus on editing the embedded LaTeX content, it is possible to change the major mode to LaTeX by typing M-x latex-mode RET. To change the major mode back to HTML+, type M-x mhtml-mode RET.

Notes from Mastering Emacs Meetups

2022-12-17T00:00:00Z

Notes from Mastering Emacs Meetups

The following notes were taken while discussing the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

Note that these notes are not a good substitute for the actual book. Not everything in the book is captured in these notes. While some points mentioned in the book are missing from the notes, there are also some points from the book where these notes discuss them in much greater detail than the book does.

Read on website | #emacs | #technology | #book | #meetup

Notes on Mastering Emacs: Chapter 1: Introduction

2022-12-16T00:00:00Z

The following notes were taken while discussing Chapter 1 of the book Mastering Emacs by Mickey Petersen (2022 edition) in book discussion group meetings.

An index of notes for all chapters are available at notes.html.

Describe System
Language Servers
Baroque Terminology
Active, Friendly Community
Vim Emulation Layer
Links

Describe System

To get a glimpse of the powerful describe system available in Emacs, type the following key sequence: C-h C-k C-p. This shows the documentation associated with the keys sequence C-p that invokes the command named previous-line to move the cursor to the previous line. The documentation mentions that this command is defined in a file named simple.el. Click on the file name to jump straight into the source code of simple.el where this command is defined. With Emacs, the source code of Emacs is not hidden from the user. The describe system is discussed in Chapter 3 of the book.

Language Servers

The two popular language server implementations for Emacs are LSP Mode and Eglot. Since Emacs 29, Eglot is part of Emacs core and does not require installing a separate package. It can be run with the key sequence M-x eglot RET.

Baroque Terminology

Some examples are:

The term point to refer to the cursor.
The term yank to paste text. What can be especially confusing for users with prior Vim experience is that Vim uses the term yank to mean copying text but Emacs uses this term to mean pasting text.
An OS-level or desktop-environment-level window is known as frame in Emacs. We can split an Emacs frame into multiple panes and these panes are called windows in Emacs.

There are many more such examples. The above examples illustrate the kind of surprises one should be ready for while reading Emacs documentation. Fortunately, it is often possible to understand the meaning of these terms from the context.

Active, Friendly Community

There are three good places to hang out with other Emacs users, seek help or help others with Emacs:

Reddit r/emacs community.
Libera #emacs channel.
Matrix #emacs room.

All three places appear to be quite friendly to beginners to Emacs!

Vim Emulation Layer

The most popular Vim emulation layer these days is Evil Mode. It is often said that the best implementation of Vim is written in Emacs Lisp. Some people also claim that Evil Mode is a better Vim than Vim itself. In fact, the next chapter, i.e. Chapter 3, says the following:

Vim users are migrating to Emacs because, well, Emacs is often a better Vim than Vim.

Mastering Emacs Book Club

2022-12-15T00:00:00Z

Mastering Emacs Book Club

This meeting series is complete! There are no new meetings happening for this book. Go to the meetings main page to find out about current active meetings.

The following content on this page is an archive of the content as it appeared on the last day of meeting for this book.

Meeting times: Usually at 20:00 UTC on Fri, Sat and Sun^†

Meeting duration: 40 minutes.

Meeting link: susam.net/meet

Meeting log: 72 meetings

Reference book: Mastering Emacs by Mickey Petersen (2022 edition)

Chapter notes: Notes

Club channel: #susam:matrix.org / #susam:libera.chat^‡

Mastodon: @susam@mastodon.social

† There are some exceptions to this schedule occasionally. Join #susam:matrix.org or #susam:libera.chat or follow @susam@mastodon.social to receive schedule updates.

‡ You only need to join either the Matrix channel or the Libera channel, not both. Both channels are bridged together. If you are not an active IRC user, prefer joining the Matrix channel because it is more convenient for someone unfamiliar with IRC. For example, you can close your browser or client and your chat session will still stay alive on Matrix. You can connect back the next day and catch up with the messages. Doing that with IRC requires slightly more work such as setting up IRC bouncers etc.

The primary reference book for these meetings is Mastering Emacs written by Mickey Petersen (2022 edition). PDF and EPUB copies of this book can be purchased from masteringemacs.org.

In case you miss any meeting, you can find recordings of the meetings that occurred in the last 7 days here: twitch.tv/susampal. These recordings are deleted automatically after 7 days.

FAQ

What is this club about?

This is a hobby club with a focus on mathematics and computation. This club picks reading material about concepts and technologies that have been around for a long time and have an air of timelessness around them. See the blog post Reading Classic Computation Books for more details.
Who runs this club?

My name is Susam. This is my website. I run this club. I host the book club meetings. The last series of book club meetings I hosted was about analytic number theory. Some members of Libera IRC network and Hacker News participated in those meetings. This new series is going to be about Emacs.
What is planned for the next few meetings?

See the meeting log which contains a rough plan for the next few meetings along with an archive of all previous meetings.
Do I need to read the planned chapters/pages in advance before coming to the meetings?

Not at all. It is up to you, really. If you would like to read the chapters in advance and come, that's great. But it is not necessary. We are going to discuss every page of the book in detail anyway. Further, we will demonstrate the concepts we learn from the book live using an actual Emacs editor. So it is perfectly fine if you want to join the meetings without any preparation.
I have little to no knowledge of Emacs. Can I join?

Yes, certainly! The reference book selected for these meetings teaches Emacs from scratch. No prior knowledge is expected from the participants of these meeetings.
I am an expert in Emacs. Will the meetings be useful to me?

In online discussions, often people with many years of Emacs experience have claimed that despite their high degree of expertise in Emacs, the book taught them new stuff. Whether attending 40-minute meetings, a few days a week, with a possibility of learning something new is a good return of your investment or not, is something you might want to consider.

Having said that, expert Emacs users and developers are very welcome to these meetings. In the previous meetings on analytic number theory, we found that the experts often contributed to the meetings with new insights and interesting background information that helped everyone learn the reading material better.
Can I just lurk in the meetings?

Yes! Lurking is absolutely fine in our club meetings. In fact, most members of the club join in and stay silent throughout the meetings. Only a few members talk via audio/video or chat. This is considered absolutely normal in this club, so please do not hesitate to join our meetings!
Tell me honestly. Which is better? Vim or Emacs?

Sorry, we don't do this here!
I have more questions. Where can I ask?

Join the club channel at #susam:matrix.org or #susam:libera.chat to ask more questions.

Read on website | #emacs | #technology | #book | #meetup

Final IANT Meeting Today

2021-10-01T00:00:00Z

Introduction

We have been reading the book Introduction to Analytic Number Theory by Apostol (1976) since March 2021. It has been going consistently since then and the previous few posts on this blog provide an account of how this journey has been so far. After about seven months of reading this book together, we are having our final meeting for this book today. This is going to be the 120th meeting of our book discussion group. The meeting notes from all previous reading sessions are archived at IANT Notes. We will discuss the final two pages of this book today and complete reading this book.

In the meeting today, we will look at some applications of the recursion formula related to partition functions that we learnt earlier. Here is an excerpt from the book that shows a specific example that demonstrates the richness and beauty of concepts one can discover while studying analytic number theory:

Equation (24) becomes \[ np(n) = \sum_{k=1}^n \sigma(k) p(n - k). \] a remarkable relation connecting a function of multiplicative number theory with one of additive number theory.

Now what equation (24) contains is not important for this post. Of course, you can refer to the book if you really want to know what equation (24) is. We learnt to prove that equation in the penultimate meeting for this subject yesterday. In this post, I will emphasise how indeed this equation is remarkable.

Introduction
The Divisor Sum Function
The Unrestricted Partition Function
The Linkage of Two Theories
The Final Meeting
Thanks!

The Divisor Sum Function

The divisor sum function $ \sigma(n) $ represents the sum of all positive divisors of $ n. $ Here are some examples: \begin{align*} \sigma(1) &= 1, \\ \sigma(2) &= 1 + 2 = 3, \\ \sigma(3) &= 1 + 3 = 4, \\ \sigma(4) &= 1 + 2 + 4 = 7, \\ \sigma(5) &= 1 + 5 = 6. \end{align*} We have spent a good amount of time with this function in the initial chapters of the book. However, for the purpose of this blog post, the definition and the examples above are good enough.

The Unrestricted Partition Function

The $ p(n) $ function is the unrestricted partition function. It represents the number of ways $ n $ can be written as a sum of positive integers $ \le n. $ Further, we let $ p(0) = 1. $ Here are some examples: \begin{align*} p(1) &= 1, \\ p(2) &= 2, \\ p(3) &= 3, \\ p(4) &= 4, \\ p(5) &= 7. \end{align*} Let me illustration the last value. The integer $ 5 $ can be represented as a sum of positive integers $ \le 5 $ in 7 different ways. They are: $ 5, $ $ 4 + 1, $ $ 3 + 2, $ $ 3 + 1 + 1, $ $ 2 + 2 + 1, $ $ 2 + 1 + 1 + 1 $ and $ 1 + 1 + 1 + 1 + 1. $ Thus $ p(n) = 5. $

The Linkage of Two Theories

The divisor sum function comes from multiplicative number theory. The partition function comes from additive number theory. Yet these two very different things get linked together in the formula mentioned in the excerpt included above. Here is the formula once again: \[ np(n) = \sum_{k=1}^n \sigma(k) p(n - k). \] How beautiful! How nicely the divisor sum function and the unrestricted partition function appear together elegantly in a single equation! Further, this equation provides a recursion formula for the partition function. Here is an illustration of this equation with $ n = 5 $: \[ 5 \cdot p(5) = 5 \cdot 7 = 35. \] \begin{align*} \sum_{k=1}^5 \sigma(k) p(5 - k) &= \sigma(1) p(4) + \sigma(2) p(3) + \sigma(3) p(2) + \sigma(4) p(1) + \sigma(5) p(0) \\ &= (1)(5) + (3)(3) + (4)(2) + (7)(1) + (6)(1) \\ &= 5 + 9 + 8 + 7 + 6 \\ &= 35. \end{align*} We will go through this topic once more in the meeting today, so if you are interested to see this formula worked out in a step-by-step manner, do join our final meeting for this book.

The Final Meeting

The final meeting is coming up at 17:00 UTC today. Visit the analytic number theory page to get the meeting link. This is not going to be the final meeting for our overall book discussion group though. This is going to be the finally meeting for only the analytic number theory book. We will have more meetings for another book after a short break.

The meeting today is going to be a lightweight session. The last two pages that we will discuss today contain some examples of recursion formulas and some commentary about Ramanujan's partition identities. Most of it should make sense even to those who have not been part of our meetings earlier, so everyone is welcome to join this meeting today, even if only to lurk. You can also join our group by joining our IRC channel where we will publish updates about future meetings. Our channel details are available in the main page here.

Thanks!

A big thank you to the Hacker News community and the Libera IRC mathematics and algorithms communities who showed interest in these meetings, joined the meetings and made this series of meetings successful.

Read on website | #mathematics | #number-theory | #meetup

Journey to Integer Partitions

2021-09-18T00:00:00Z

Introduction

After 114 meetings and 75 hours of studying together, our analytic number theory book discussion group has finally reached the final chapter of the book Introduction to Analytic Number Theory by Apostol (1976). We have less than 18 pages to read in order to complete reading this book. Considering that we meet 3-4 times in a week and we discuss about 2-3 pages in every meeting, it appears that we would be able to complete reading this book in another 2 weeks.

Reading this book has been quite a journey! The previous three posts on this blog provide an account of how this journey has been. It has been fun, of course. The best part of hosting a book discussion group like this has been the number of extremely smart people I got an opportunity to meet and interact with. The insights and comments on the study material that others shared during the meetings were very helpful.

The meeting log shows that our meetings started really small with only 4 participants in the first meeting in March 2021 and then it gradually grew to about 10-12 regular members within a month. Then a few months later, the number of participants began dwindling a little. This happened because some members of the group had to drop out as they got busy with other personal or professional engagements. However, six months later, we still have about 4-5 regular participants meeting consistently. I think it is pretty good that we have made it this far.

Unrestricted Partitions

The final chapter on integer partitions is very unlike all the previous 12 chapters. While the previous chapters dealt with multiplicative number theory, this final chapter deals with additive number theory. For example, the first theorem talks about an interesting property of unrestricted partitions. We study the number of ways a positive integer can be expressed as a sum of positive integers. The number of summands is unrestricted, repetition of summands is allowed and the order of the summands is not taken into account. For example, the number 3 has 3 partitions: 3, 2 + 1 and 1 + 1 + 1. Similarly, the number 4 has 5 partitions: 4, 3 + 1, 2 + 2, 2 + 1 + 1 and 1 + 1 + 1 + 1.

I have always wanted to learn about partitions more deeply, so I am quite happy that this book ends with a chapter on partitions. The subject of partitions is rich with very interesting results obtained by various accomplished mathematicians. In the book, the first theorem about partitions is a very simple one that follows from the geometric representation of partitions. Let us see an illustration first.

How many partitions of 6 are there? There are 11 partitions of 6. They are 6, 5 + 1, 4 + 2, 4 + 1 + 1, 3 + 3, 3 + 2 + 1, 3 + 1 + 1 + 1, 2 + 2 + 2, 2 + 2 + 1 + 1, 2 + 1 + 1 + 1 + 1 and 1 + 1 + 1 + 1 + 1 + 1. Now how many of these partitions are made up of 5 parts? Each summand is called a part. The answer is 2. There are 2 partitions of 6 that are made up of 5 parts. They are 3 + 1 + 1 + 1 and 2 + 2 + 1 + 1. Let us represent both these partitions as arrangements of lattice points. Here is the representation of the partition 3 + 1 + 1 + 1:

• • •
•
•
•

Now if we read this arrangement from left to right, column by column, we get another partition of 6, i.e. 4 + 1 + 1. Note that the number of parts in 3 + 1 + 1 + 1 (i.e. 4) appears as the largest part in 4 + 1 + 1. Similarly, the number of parts in 4 + 1 + 1 (i.e. 3) appears as the largest part in 3 + 1 + 1 + 1. Let us see one more example of this relationship. Here is the geometric representation of 2 + 2 + 1 + 1:

• •
• •
•
•

Once again, reading this representation from left to right, we get 4 + 2, another partition of 6. Once again, we can see that the number of partitions in 2 + 2 + 1 + 1 (i.e. 4) appears as the largest part in 4 + 2 and vice versa. These observations lead to the first theorem in the chapter on partitions:

Theorem 14.1 The number ofpartitions of $ n $ into $ m $ parts is equal to the number of partitions of $ n $ into parts, the largest of which is $ m. $

That was a brief introduction to the chapter on partitions. In the next two or so weeks, we will dive deeper into the theory of partitions.

Next Meeting

If this blog post was fun for you, consider joining our next meeting. Our next meeting is on Tue, 21 Sep 2021 at 17:00 UTC. Since we are at the beginning of a new chapter, it is a good time for new participants to join us. It is also a good time for members who have been away for a while to join us back. Since this chapter does not depend much on the previous chapters, new participants should be able to join our reading sessions for this chapter and follow along easily without too much effort.

To join our discussions, see our channel details in the main page here. To get the meeting link for the next meeting, visit the analytic number theory book page.

It is worth mentioning here that lurking is absolutely fine in our meetings. In fact, most participants of our meetings join in and stay silent throughout the meeting. Only a few members talk via audio/video or chat. This is considered absolutely normal in our meetings, so please do not hesitate to join our meetings!

Read on website | #mathematics | #number-theory | #meetup

Journey to the Prime Number Theorem

2021-09-09T00:00:00Z

How long does it take to start with zero knowledge of analytic number theory and successfully learn the analytic proof of the prime number theorem? Take a guess! I will share my answer in the next two paragraphs. This is something I had wondered when we began our analytic number theory book discussion group in March 2021. Back then, I thought it would take at least 100 hours of effort.

The book I had chosen for our discussions was Introduction to Analytic Number Theory by Apostol (1976). I have been hosting 40-minute meetings for about 3-4 days every week since March 2021. We discuss a couple of pages of the book in every meeting. Most participants in this meeting are from Hacker News and Libera IRC network. For a long time, I was eager to learn the proof of the prime number theorem. For those unfamiliar with the theorem, I will describe it briefly in further sections. Let me first answer the question I asked in the previous paragraph.

So how long does it take to start with no knolwedge of analytic number theory and teach ourselves the analytic proof of the prime number theorem? Turns out, it takes 72 hours! It took our group 72 hours spread across 110 meetings over 6 months to be able to understand the proof. It is worth noting here that most of us in this group have full-time jobs and other personal obligations! We were all doing this for fun, for the joy of learning!

Now I must mention that the 72 hours noted above is only the time spent together in reading the book and working through the theorems and proofs. It does not include the personal time spent in solving problems, reading some sections again, taking notes, etc. All of that was done in our personal time. We did discuss the solutions to some of the very interesting problems in our meetings just to take a break from the theorem-and-proof style of reading but most of these 72 hours of meetings focussed on working through the theorems and proofs in the book.

It may be possible to achieve this milestone in lesser number of hours, perhaps by reading the book alone which for some folks might be faster than studying in a group or perhaps by skipping some chapters for topics that look very familiar. In our discussions, however, we did not skip any chapter. There were in fact a few chapters we could have skipped. All members of these meetings were very familiar with divisibility, greatest common divisor, the fundamental theorem of arithmetic, etc. discussed in Chapter 1. Most of us were also very familiar with the concepts discussed in Chapter 5 such as congruences, residue classes, the Euler-Fermat theorem, the Chinese remainder theorem, etc. Despite being familiar with these concepts, we decided not to skip any chapter for the sake of completeness of our coverage of the material. In fact, we read every single line of the book and deliberated over every single concept discussed in the book. With this detailed and tedious approach to reading the book, it took us 72 hours to read about 290 pages and learn the analytic proof of the prime number theorem in Chapter 13.

Prime Number Theorem
Equivalent Forms
Dirichlet, Dirichlet, Dirichlet!
Chain of Proofs
Conclusion

Prime Number Theorem

The prime number theorem is a very curious fact about the distribution of prime numbers that Gauss noticed in the year 1792 when he was about 15 years old. He noticed that the occurrence of primes become rarer and rarer as we expand our search for them to larger and larger integers. For example, there are 4 primes between 1 and 10, i.e. 40% of the numbers between 1 and 10 are primes! But there are only 25 primes between 1 and 100, i.e. only 25% of the numbers between 1 and 100 are primes. If we go up to 1000, we notice that there are only 168 primes between 1 and 1000, i.e. only 16.8% of the numbers between 1 and 1000 are primes. Formally, we denote these facts with the mathematical notation $ \pi(x) $ that denotes the prime counting function. We say $ \pi(10) = 4, $ $ \pi(100) = 25, $ $ \pi(1000) = 168 $ and so on. Note that we allow $ x $ to be a real number, so while $ \pi(10) = 4, $ we have $ \pi(10.3) = 4 $ as well. One of the reasons we let $ x $ be a real number in the definition of $ \pi(x) $ is because it makes various problems we come across during the study of this function more convenient to work on using real analysis.

We observe that the 'density' of primes continue to fall as we make $ x $ larger and larger. In formal notation, we see that the ratio $ \pi(x) / x $ is $ 0.4 $ when $ x = 10. $ This ratio falls to $ 0.25 $ when $ x = 100. $ It falls further to $ 0.168 $ when $ x = 1000 $ and so on. Can we predict by how much this "density" falls? The answer is yes. That leads us to the prime number theorem. The prime number theorem states that $ \pi(x) / x $ is asymptotic to $ 1 / \log x $ as $ x $ approaches infinity, i.e. \[ \frac{\pi(x)}{x} \sim \frac{1}{\log x} \text{ as } x \to \infty. \] For those unfamiliar with the notation of asymptotic equality, here is another equivalent way to state the above relationship, \[ \lim_{x \to \infty} \frac{\pi(x) / x}{1 / \log x} = 1. \] We could also write this as \[ \lim_{x \to \infty} \frac{\pi(x)}{x / \log x} = 1 \] or \[ \pi(x) \sim \frac{x}{\log x} \text{ as } x \to \infty. \] Let us see how well this formula works as an estimate for the density of primes for small values of $ x. $

$ x $	$ \pi(x) $	$ x / \log x $
10	4	4.3
100	25	21.7
1000	168	144.8
10000	1229	1085.7
100000	9592	8685.9

Not bad! In fact, the last two columns begin to agree more and more as $ x $ becomes larger and larger.

The analytic proof of the prime number theorem was achieved with an intricate chain of equivalences and implications between various theorems. The book consumes 13 chapters and 290 pages before completing the proof of the prime number theorem. Each page is also quite dense with information. The amount of commentary or illustrations is very little in the book. Most of the book keeps alternating between theorem statements and proofs. Occasionally, for especially long chapters with an intricate sequence of proofs, Apostol provides a plan of the proof in the introductions to such chapters. It is quite hard to summarise a large and dense volume of work like this in a blog post but I will make an attempt to paint a very high-level picture of some of the key concepts that are involved in the proof.

Equivalent Forms

Everything from Chapters 1 to 3 is about building basic concepts and tools we will use later to work on the problem of the prime number theorem. These concepts and tools were very interesting on their own. They involved divisibility, various number-theoretic functions, Dirichlet products, the big oh notation, etc. Chapter 4 was the first chapter where we engaged ourselves with the prime number theorem. This chapter taught us several other formulas that were logically equivalent to the prime number theorem. One equivalence that would play a big role later was the equivalence between the prime number theorem \[ \lim_{x \to \infty} \frac{\pi(x) \log x}{x} = 1 \] and the following form: \[ \lim_{x \to \infty} \frac{\psi(x)}{x} = 1. \] If we could prove one, the validity of the other would be established automatically. The notation $ \psi(x) $ denotes the Chebyshev function which in turn is defined in terms of the Mangoldt function $ \Lambda(n) $ as $ \psi(x) = \sum_{n \le x} \Lambda(n). $ Note that the formula above can also be stated using the asymptotic equality notation as follows: \[ \psi(x) \sim x \text{ as } x \to \infty. \] There were several other equivalent forms too shown in Chapter 4. The fact that all these various forms were equivalent to each other was rigorously proved in the chapter. Thus proving any one of the equivalent forms would be sufficient to prove the prime number theorem. But in Chapter 4, we did not know how to prove any of the equivalent forms. We could only prove the equivalence of the various formulas, not the formulas themselves. We only learnt that if any of the equivalent forms is true, so is the prime number theorem. Similarly, if any of the equivalent forms is false, so is the prime number theorem. We would visit the prime number theorem again in Chapter 13 which would complete the proof of the prime number theorem by showing that the equivalent form mentioned above is indeed true.

Dirichlet, Dirichlet, Dirichlet!

Chapters 5 to 10 introduced more concepts involving congruences, finite abelian groups, their characters, Dirichlet characters, Dirichlet's theorem on primes in arithmetic progressions, Gauss sums, quadratic residues, primitive roots, etc. Some of these concepts would turn out to be very important in proving the prime number theorem but most of them probably are not too important if understanding the proof of the prime number theorem is the only goal. Regardless, all of these chapters were very interesting.

It was in Chapters 11 and 12 that we felt that we were getting closer and closer to the proof of the prime number theorem. Chapter 11 began a detailed and rigorous study of convergence and divergence of Dirichlet series. The Riemann zeta function is a specific type of Dirichlet series. Chapter 12 introduced analytic continuation of the Riemann zeta function. We could then show interesting results like $ \zeta(0) = -1/2 $ and $ \zeta(-1) = -1/12 $ using the analytic continuation of the zeta function. This chapter also showed us why all trivial zeroes of $ \zeta(s) $ must lie at negative even integers.

One thing I realised during the study of this book is how frequently we use concepts, operations, functions and theorems named after Dirichlet. It was impossible to get through a meeting without having uttered "Dirichlet" at least a dozen times!

Chain of Proofs

Finally, Chapter 13 showed us how to prove the prime number theorem. The plan of the proof was laid out in the first section. Our goal in this chapter is to prove that $ \psi(x) \sim x $ as $ x \to \infty. $ This is equivalent to the prime number theorem, so proving this amounts to proving the prime number theorem too.

Next we learn that the asymptotic relation $ \psi_1(x) \sim x^2 / 2 $ as $ x \to \infty $ implies the previous asymptotic relationship. Here $ \psi_1(x) $ is defined as $ \psi_1(x) = \int_1^x \psi(t) \, dt. $ This implication is proved quite easily in one and a half pages. But we still need to show that the asymptotic relation $ \psi_1(x) \sim x^2 / 2 $ as $ x \to \infty $ indeed holds good. Proving this takes a lot of work. To prove this asymptotic relation we first learn to arrive at the following equation involving a contour integral: \[ \frac{\psi_1(x)}{x^2} - \frac{1}{2} \left( 1 - \frac{1}{x} \right)^2 = \frac{1}{2\pi i} \int_{c - \infty i}^{c + \infty i} \frac{x^{s - 1}}{s(s + 1)} \left( -\frac{\zeta'(s)}{\zeta(s)} - \frac{1}{s - 1} \right) \, ds \] for $ c \gt 1. $ The equation above looks quite complex initially but each part of it becomes friendly as we learn to derive it and then work on each part of it while working out further proofs. Now if we could somehow show that the integral on the right hand side of the above equation approaches 0 as $ x \to \infty, $ that would end up proving the asymptotic relation involving $ \psi_1(x) $ and thus end up proving the prime number theorem by equivalence. However, proving that this integral indeed becomes 0 as $ x \to \infty $ requires a careful study of $ \zeta(s)/\zeta'(s) $ in the vicinity of the line $ \operatorname{Re}(s) = 1. $ This is the topic that most of the chapter deals with.

This plan of the proof looked quite convoluted initially but Apostol has done a great job in this chapter to first walk us through this plan and then prove each fact that we need to make the proof work in a detailed and rigorous manner. When we reached the end of the proof, one of our regular members remarked, "Now the proof does not look so complex!"

Would the elementary proof of the prime number theory have been easier? I don't know. I have not studied the elementary proof. But Apostol does say this at the beginning of Chapter 13,

The analytic proof is shorter than the elementary proof sketched in Chapter 4 and its principal ideas are easier to comprehend.

Learning the analytic proof itself was quite a long journey that required dedication and consistency in our studies over a period of 6 months. If we trust the above excerpt from the book, then I think it is fair to assume that the elementary proof is even more formidable.

Conclusion

That was an account of our journey through an analytic number theory book from its first chapter up to the analytic proof of the prime number theorem. We have not completed reading the entire book though. We still have about another 30 pages to go through. In the remaining study of this book, we will learn more about zero-free regions for $ \zeta(s), $ the application of the prime number theorem to the divisor function and the Euler totient function. The next and the final chapter too has a lot to offer such as integer partition, Euler's pentagonal-number theorem and the partition identities of Ramanujan. I am pretty hopeful that we will be complete reading this book in another few weeks of meetings.

Read on website | #mathematics | #number-theory | #meetup

One Hundred Meetings

2021-08-20T00:00:00Z

Today, our computation book discussion group is going to have the 100th meeting! Yes, the 100th meeting! We began these book discussion meetings about five months ago. The first book we picked up for our discussions was Introduction to Analytic Number Theory by Apostol (1976). We have been reading this book together for the last five months. We have a tiny but consistent community of 6 to 8 participants who meet regularly to study this book and share our understanding and insights with each other.

In this blog post, I will talk about my personal experience hosting these meetings and my personal journey about reading this book. It is worth keeping in mind then that what I am about to write below may not have any resemblance with the experience of other participants of these meetings.

The Reading Experience
The Learning Experience
Three Concepts
The Next Meeting
Join Us

The Reading Experience

As far as I know, everyone who joins our meetings are involved in computer programming in one form or another. A few of them have very strong background in mathematics. I host these meetings everyday and discuss a few sections of the book in detail. I show how to work through the proofs, explain some of the steps, etc. Sometimes I get stuck in some step that I find too unobvious. Sometimes the steps are obvious but my brain is too slow to understand why the steps work. But these tiny glitches have not been a problem so far, thanks to all the members who join these meetings on a daily basis and contribute their explanations of the proofs.

I believe the group members are the best part of these discussions. Thanks to the insights and explanation of the reading material shared by all these members, I am fairly confident that we are able to take a close look at every proof and convince ourselves that every step of the proofs work.

The Learning Experience

The first web meeting to discuss the chosen analytic number theory book occurred on 5 Mar 2021. See the blog post Reading Classic Computation Books to read about the early days of our group and how it was formed. Back then, I knew little to nothing about analytic number theory. Although I was familiar with some of the elementary concepts like divisibility, Euler's totient function, modular arithmetic, calculus and related theorems, chapter 2 of the book itself proved to be a significant challenge for me. In the second chapter, it became clear to me that we will be building new levels of mathematical abstractions, use these abstractions to build yet another layer of abstractions and so on. The chapter began with a description of the Möbius function, a very neat and interesting function that I was previously unaware of. That was fun! But soon, this chapter began adding new layers of abstractions such as Dirichlet product, Dirichlet inverse, generalised convolution, etc. I could almost feel my brain stretching and growing as we went through each page of this chapter.

I often saw that after I have learnt a new concept in a chapter, it would not become intuitive immediately. I would understand the concepts, understand the related theorems, understand each step of the proofs, solve exercise problems, know how to apply the theorems when needed and yet I could not "feel" them. I wanted to not just understand the concepts but I also wanted to "feel" the concepts like the way I could feel algebra, calculus, computer programming, etc. In the initial days, I wondered if I was too old to develop good intuition for all these new and highly sophisticated concepts.

Despite always feeling that all these concepts were too technical and quite unintuitive, I kept going. I kept hosting these discussions with a frequency of about 3-5 days every week. We continued discussing the various chapters and the proofs in them. And then suddenly one day while reading chapter 4, something interesting happened. As we were employing Dirichlet products to obtain some useful results, I realised that the concept of Dirichlet products which once felt so foreign two chapters earlier, now felt completely intuitive. I could see different functions being equivalent to Dirichlet products intuitively and effortlessly. Dirichlet products felt no more alien than, say, arithmetic multiplication. I could "feel" it now. It was a great feeling. I realised that sometimes it might take a few additional chapters of reading and using those concepts over and over again before they really begin to feel intuitive.

Three Concepts

In this section, I will pick three interesting concepts from different parts of the book to provide a glimpse of what the journey has been like. These three things occur in the book again and again and play a very important role in several chapters of the book. Of course, it goes without saying that there are many interesting concepts in the book and many of them may be more important than the ones I am about to show below.

The Möbius Function

For any positive integer $ n, $ the Möbius function $ \mu(n) $ is defined as follows: \[ \mu(1) = 1; \] If $ n \gt 1, $ write $ n = p_1^{a_1} \dots p_k^{a_k} $ (prime factorisation). Then \begin{align*} \mu(n) & = (-1)^k \text{ if } a_1 = a_2 = \dots = a_k = 1, \\ \mu(n) & = 0 \text{ otherwise}. \end{align*} If $ n \ge 1, $ we have \[ \sum_{d \mid n} \mu(d) = \begin{cases} 1 & \text{ if } n = 1, \\ 0 & \text{ if } n \gt 1. \end{cases} \]

I was unfamiliar with this function prior to reading the book. It felt like a nice little cute function initially but as we went through more chapters, it soon became clear that this function plays a major role in analytic number theory.

As a simple example, we will soon see in this post that the Euler's totient function can be expressed as a Dirichlet product of the Möbius function and the arithmetical function $ N(n) = n. $

As a more sophisticated example, the Dirichlet series with coefficients as the Möbius function is the multiplicative inverse of the Riemann zeta function, i.e. if $ s = \sigma + it $ is a complex number with its real part $ \sigma \gt 1, $ we have \[ \sum_{n=1}^{\infty} \frac{\mu(n)}{n^s} = \frac{1}{\zeta(s)}. \] This immediately shows that $ \zeta(s) \ne 0 $ for $ \sigma \gt 1. $

Dirichlet Product

If $ f $ and $ g $ are two arithmetical functions, their Dirichlet product $ f * g $ is defined as: \[ (f * g)(n) = \sum_{d \mid n} f(d) g\left( \frac{n}{d} \right). \] Dirichlet products appear to pop up magically at various places in number theory. Here is a simple example: \[ \varphi(n) = \sum_{d \mid n} \mu(d) \frac{n}{d}. \] Therefore in the notation of Dirichlet products, the above equation can also be written as \[ \varphi = \mu * N \] where $ N $ represents the arithmetical function $ N(n) = n $ for all $ n. $

Hurwitz Zeta Function

For complex numbers $ s = \sigma + it, $ the Hurwitz zeta function $ \zeta(s, a) $ is initially defined for $ \sigma \gt 1 $ as \[ \zeta(s, a) = \sum_{n=0}^{\infty} \frac{1}{(n + a)^s} \] where $ a $ is a fixed real number, $ 0 \lt a \lt 1. $ Then by analytic continuation, it is defined for $ \sigma \le 1 $ as \[ \zeta(s, a) = \Gamma(1 - s)I(s, a) \] where $ \Gamma $ represents the gamma function \[ \Gamma(s) = \int_0^{\infty} x^{s - 1} e^{-x} \, dx \] defined for $ \sigma \gt 0 $ and also defined, by analytic continuation, for $ \sigma \le 0 $ except for $ \sigma = 0, -1, -2, \dots $ (the nonpositive integers) and $ I(s, a) $ is defined by the contour integral \[ I(s, a) = \frac{1}{2\pi i} \int_C \frac{z^{s-1} e^{az}}{1 - e^z} \, dz \] where $ 0 \lt a \le 1 $ and the contour $ C $ is a loop around the negative real axis composed of three parts $ C_1, $ $ C_2 $ and $ C_3 $ such that for $ c \lt 2\pi, $ we have $ z = re^{-\pi i} $ on $ C_1 $ and $ z = re^{\pi i} $ on $ C_3 $ as $ r $ varies from $ c $ to $ +\infty $ and $ z = ce^{i \theta} $ on $ C_2, $ $ -\pi \le \theta \le \pi. $

Now admittedly, the definition or the analytic continuation of Hurwitz zeta function may seem very heavy and obscure to the uninitiated and it is indeed quite heavy. It takes 6 pages in chapter 12 to build the prerequisite concepts before we arrive at this definition. It is evident that this definition uses other concepts like the gamma function, a specific contour integral, etc. and it is only natural to expect that one has to gain sufficient expertise with the gamma function and contour integrals before the Hurwitz zeta function begins to feel intuitive.

But once we have established the analytic continuation of the Hurwitz zeta function, many insightful facts about the Riemann zeta function follow readily. It is easy to see that the Riemann zeta function can be defined in terms of the Hurwitz zeta function as \[ \zeta(s) = \zeta(s, 1) = \sum_{n=1}^{\infty} \frac{1}{n^s}. \] Yes, the $ \zeta $ symbol is overloaded: $ \zeta(s, a) $ is the Hurwitz zeta function whereas $ \zeta(s) $ is the Riemann zeta function. This relationship between the Riemann zeta function and the Hurwitz zeta function along with the analytic continuation of the Hurwitz zeta function opens new doors into the wonderful world of complex numbers and let us obtain beautiful and profound facts about the Riemann zeta function such as the fact that it has zeros at negative even integers, i.e. $ \zeta(n) = 0 $ for $ n = -2, -4, -6, \dots $ and the fact that $ \zeta(0) = -\frac{1}{2} $ and $ \zeta(-1) = -\frac{1}{12} $ and so on.

I believe beautiful results like these obtained by digging deep into complex analysis are what makes the study of analytic number theory so rewarding.

The Next Meeting

The next meeting is coming up today in a few hours. Are we planning anything special for the 100th meeting?

I think the 100th meeting is a significant milestone in our journey of understanding the beautiful and interesting gems hidden away in the subject of analytic number theory. This milestone has been possible only due to the sustained curiousity and eagerness among the members of the group to learn a significant area of mathematics and learn it well. We have reached this milestone successfully due to the passion and love for mathematics that drive the regular members to join these meetings and go through a few pages of the book everyday. In these meetings, we have read 12 chapters consisting of over 250 pages so far. Many of us knew nothing about analytic number theory merely five months ago and now we can appreciate the Riemann zeta function at a deeper level. We now understand what the Riemann hypothesis really means. This has been a great journey so far.

Despite being a significant milestone and cause for celebration, we are going to keep our 100th meeting fairly simple. We will continue where we left off yesterday. Today we have some more relationships between the gamma function and the Riemann zeta function to go through, so that is what we will do. We will also show that $ \zeta(0) = -\frac{1}{2} $ and $ \zeta(-1) = -\frac{1}{12} $ using the analytic continuation of the Hurwitz zeta function today.

Join Us

If this blog post was fun for you and you would like to join our meetups, please go through this page to get the meeting link and join us.

Read on website | #mathematics | #number-theory | #meetup

Reading Classic Computation Books

2021-06-15T00:00:00Z

Hello readers and learners! In the last few months, some of us from mathematics and algorithms communities of IRC networks have come together to form a tiny group that meets a few times every week and reads a book together. The focus of this loosely formed group is on picking reading material about concepts and technologies that have been around for a long time and have an air of timelessness around them.

In particular, we will not be chasing the latest trends in technology most of the time. But this is not a retrocomputing community either. For example, we are likely not going to talk about BASIC programming on Commodore 64. The topics we pick for our discussions are largely going to be topics that are still quite relevant and important in the field of mathematics and computation but they may not be what is considered to be mainstream. For example, we will read books like Introduction to Analytic Number Theory or talk about Common Lisp programming. Sometimes these topics may intersect with modern trends or retrocomputing but most of the times they would not. That is the kind of sweet spot we want to hit.

Origin

Here is a little bit of history about the community around these discussions. This community was born on the Freenode IRC network when I announced on the ##algorithms and ##math channels there that I would be hosting book reading and discussion meetings on analytic number theory. I created a makeshift channel named #susam to discuss the book and plan the meeting schedule. Several members joined this channel and we were ready to kick off our first meeting.

Our first meeting occurred via IRC on 4 Mar 2021. We realised that it would be much more efficient to discuss mathematics via audio along with a shared desktop session where we can use some software tool to scribble out and render mathematics formulas using LaTeX. So on the next day, that is, on 5 Mar 2021, we had our first web meeting. Later, after an announcement on Hacker News, some members from the Hacker News community joined our channel too and the community grew a little bit more as a result.

Present

We moved to Matrix later and now continue to operate out of the #susam:matrix.org. It has now been more than 3 months since we began these meetings. Although our community is tiny, the meetings have been going on consistently and we have learnt numerous new and interesting concepts and theorems in the field of analytic number theory. In this community, we pick classics from the field of mathematics and computation and read them together.

Since this group was formed, we have made good progress with the study of analytic number theory. The meetings for this topic have been going on consistently. When I get some more free time now in future, I will add a few more topics for our meetings.

Join Us

Join the discussion channel at #susam:matrix.org. and be a part of these activities. See you there!

Read on website | #meetup

Notes on Chapter 3: Averages of Arithmetical Functions

2021-04-09T00:00:00Z

§ 3.3: Euler's summation formula

Main idea behind the proof of Euler's summation formula

Important numbers in the proof: \[ 0, \quad \underbrace{[y]}_{=\,m}, \quad y, \quad \underbrace{[y] + 1}_{=\,m + 1}, \quad \underbrace{[x]}_{=\,k}, \quad x. \] Splitting the definite integral: \[ \int_y^x f(t)\,dt = \int_{y}^{[y] + 1} f(t)\,dt + \underbrace{\int_{[y] + 1}^{[y] + 2} f(t)\,dt + \dots + \int_{[x] - 1}^{[x]} f(t)\,dt}_{=\,\int_{[y] + 1}^{[x]} f(t)\, dt} + \int_{[x]}^{x} f(t)\,dt. \] Using the more convenient variables $ m $ and $ k, $ we get: \[ \int_y^x f(t)\,dt = \int_m^{m + 1} f(t)\,dt + \underbrace{\int_{m + 1}^{m + 2} f(t)\,dt + \dots + \int_{k - 1}^{k} f(t)\,dt}_{=\,\int_{m + 1}^{k} f(t)\, dt} + \int_{k}^{x} f(t)\,dt. \]

Sum of integrals in the proof Euler's summation formula

\begin{align*} \int_{m + 1}^{k} [t] f'(t) dt & = \int_{m + 1}^{m + 2} [t] f'(t) dt + \int_{m + 2}^{m + 3} [t] f'(t) dt + \dots + \int_{k - 1}^{k} [t] f'(t) dt \\ & = \begin{aligned}[t] & (m + 2) f(m + 2) - (m + 1) f(m + 1) - f(m + 2) \\ + & (m + 3) f(m + 3) - (m + 2) f(m + 2) - f(m + 3) \\ & \dots \\ + & (k) f(k) - (k - 1) f(k - 1) - f(k) \end{aligned} \\ & = kf(k) - (m + 1)f(m + 1) - \sum_{n=m + 2}^{k} f(n) \\ & = kf(k) - mf(m + 1) - f(m + 1) - \sum_{n=m + 2}^{k} f(n) \\ & = kf(k) - mf(m + 1) - \sum_{n=m + 1}^{k} f(n) \\ & = kf(k) - mf(m + 1) - \sum_{y \lt n \le x} f(n). \end{align*}

Equation (6) in the proof of Euler's summation formula

\begin{align*} \sum_{y \lt n \le x} f(n) & = - \int_{m + 1}^k [t] f'(t) \, dt + k f(k) - m f(m + 1) \\ & = \begin{aligned}[t] & \left( - \int_y^{m + 1} [t] f'(t) \, dt - \int_{m + 1}^k [t] f'(t) \, dt - \int_k^x [t] f'(t) \, dt \right) \\ & + f(k) - m f(m + 1) + \int_y^{m + 1} [t] f'(t) \, dt + \int_k^x [t] f'(t) \, dt \end{aligned} \\ & = - \int_y^x [t] f'(t) \, dt + k f(k) - m f(m + 1) + \int_y^{m + 1} m f'(t) \, dt + \int_k^x k f'(t) \, dt \\ & = - \int_y^x [t] f'(t) \, dt + k f(k) - m f(m + 1) + \biggl( m f(m + 1) - m f(y) \biggr) + \biggl( k f(x) - k f(k) \biggr) \\ & = - \int_y^x [t] f'(t) \, dt + k f(x) - m f(y). \end{align*}

Using integration by parts in the proof of Euler's summation formula

Integration by parts: \[ \int uv \, dt = u \int v \, dt - \int u' \left( \int v \, dt \right) \, dt. \] \[ \int_y^x t f'(t) \, dt = \left. \left( t f(t) - \int f(t) \, dt \right) \right|_y^x = x f(x) - y f(y) - \int_y^x f(t) \, dt. \] Final step of the proof: \begin{align*} \sum_{y \lt n \le x} f(n) & = -\int_y^x [t] f'(t) \, dt + k f(x) - m f(y) \\ & = \begin{aligned}[t] & -\int_y^x [t] f'(t) \, dt + [x] f(x) - [y] f(y) \\ & + \underbrace{ \left( \int_y^x t f'(t) \, dt - x f(x) + y f(y) + \int_y^x f(t) \, dt \right)}_{0 \text{ by above definite integral}} \end{aligned} \\ & = \int_y^x f(t) \, dt + \int_y^x (t - [t]) f'(t) \, dt + f(x)([x] - x) - f(y)([y] - y). \end{align*}

§ 3.3: Some elementary asymptotic formulas

Splitting integral in the proof of Theorem 3.2

Splitting definite integral: \begin{align*} & \int_1^{\infty} f(t) \, dt = \int_1^{x} f(t) \, dt + \int_x^{\infty} f(t) \, dt \\ & \iff \int_1^{\infty} f(t) \, dt - \int_x^{\infty} f(t) \, dt = \int_1^x f(t) \, dt. \end{align*} Solving improper integral: \[ \int_x^{\infty} \frac{1}{t^2} \, dt = \lim_{b \to \infty} \int_x^b \frac{1}{t^2} dt = \lim_{b \to \infty} \frac{-1}{t} \Biggr|_x^b = \left( \lim_{b \to \infty} \frac{-1}{b} \right) + \frac{1}{x} = 0 + \frac{1}{x} = \frac{1}{x}. \]

Euler's constant in the proof of Theorem 3.2 (a)

Definition of Euler's constant: \[ C = \lim_{n \to \infty} \left( 1 + \frac{1}{2} + \frac{1}{3} + \dots + \frac{1}{n} - \log n \right) = \lim_{x \to \infty} \left( \sum_{n \le x} \frac{1}{n} - \log x \right). \] We begin with \[ \sum_{n \le x} \frac{1}{n} = \log x + \underbrace{1 - \int_1^{\infty} \frac{t - [t]}{t^2} \, dt}_{\text{We will show below that this is $ C $}} + O\left( \frac{1}{x} \right). \] Rearranging the terms, we get \[ \sum_{n \le x} \frac{1}{n} - \log x = 1 - \int_1^{\infty} \frac{t - [t]}{t^2} \, dt + O\left( \frac{1}{x} \right). \] Using the definition of $ C, $ we get \begin{align*} C & = \lim_{x \to \infty} \left( \sum_{n \le x} \frac{1}{n} - \log x \right) \\ & = \lim_{x \to \infty} \left( 1 - \int_1^{\infty} \frac{t - [t]}{t^2} \, dt + O\left( \frac{1}{x} \right) \right) \\ & = 1 - \int_1^{\infty} \frac{t - [t]}{t^2} \, dt. \end{align*}

Some integrals in the proof of Theorem 3.2 (b)

\[ \int_1^x \frac{dt}{t^s} = \frac{t^{-s + 1}}{-s + 1} \Biggr|_1^x = \frac{t^{1 - s}}{1 - s} \Biggr|_1^x = \frac{x^{1 - s}}{1 - s} - \frac{1}{1 - s}. \] \[ \int_1^x \frac{t - [t]}{t^{s + 1}} \, dt = \int_1^{\infty} \frac{t - [t]}{t^{s + 1}} \, dt - \int_x^{\infty} \frac{t - [t]}{t^{s + 1}} \, dt = \int_1^{\infty} \frac{t - [t]}{t^{s + 1}} \, dt + \underbrace{\frac{1}{s} O\left( x^{-s}\right)}_{\text{explained below}}. \] \[ 0 \le \int_x^{\infty} \frac{t - [t]}{t^{s + 1}} \, dt \le \int_x^{\infty} \frac{1}{t^{s + 1}} \, dt = \frac{-1}{st^s} \Biggr|_x^\infty = \frac{1}{sx^s} = \frac{1}{s} x^{-s}. \] \begin{align*} \sum_{n \le x} \frac{1}{n^s} & = \int_1^x \frac{dt}{t^s} - s \int_1^x \frac{t - [t]}{t^{s + 1}} + 1 - \frac{x - [x]}{x^s} \, dt \\ & = \frac{x^{1 - s}}{1 - s} - \frac{1}{1 - s} - s \int_1^{\infty} \frac{t - [t]}{t^{s + 1}} \, dt + 1 + O(x^{-s}). \end{align*}

Read on website | #mathematics | #number-theory | #book | #meetup

Notes on Chapter 2: Arithmetical Functions and Dirichlet Multiplication

2021-03-26T00:00:00Z

§ 2.11: The inverse of a completely multiplicative function

Completely Multiplicative Function

\[ f(mn) = f(m) f(n) \text{ for all } m, n. \]

Identity function (Dirichlet product)

\[ I(n) = \begin{cases} 1 & \text{ if } n = 1, \\ 0 & \text{ if } n \gt 1. \end{cases} \]

\begin{align*} f(n)I(n) & = \begin{cases} 1 \cdot 1 & \text{ if } n = 1, \\ f(n) \cdot 0 & \text{ if } n \gt 1. \end{cases} \\ & = \begin{cases} 1 & \text{ if } n = 1, \\ 0 & \text{ if } n \gt 1. \end{cases} \\ & = I(n). \end{align*}

Möbius function for prime powers

\[ \mu(1) = 1, \qquad \mu(p) = -1, \qquad \mu(p^2) = \mu(p^3) = \dots = 0. \]

Third equation in the proof of Theorem 2.17

\begin{align*} \sum_{d \mid p^a} \mu(d) f(d) f\left(\frac{p^a}{d}\right) & = \sum_{d = 1, p, p^2, \dots, p^a} \mu(d) f(d) f\left(\frac{p^a}{d}\right) \\ & = \begin{aligned}[t] & \mu(1) f(1) f\left( \frac{p^a}{1} \right) + \mu(p) f(p) f\left( \frac{p^a}{p} \right) \\ & + \underbrace{\mu(p^2) f(p^2) f\left( \frac{p^a}{p^2} \right) + \dots + \mu(p^a) f(p^a) f\left( \frac{p^a}{p^a} \right)}_{=\,0} \end{aligned} \\ & = \mu(1) f(1) f(p^a) + \mu(p) f(p) f(p^{a - 1}) \\ & = f(p^a) - f(p) f(p^{a - 1}). \end{align*}

Final step in the proof of Theorem 2.17

\begin{align*} f(p^a) & = f(p)f(p^{a - 1}) \\ & = f(p)f(p)f(p^{a - 2}) \\ & = \dots \\ & = \underbrace{f(p)f(p)f(p) \dots f(p)}_{a \text{ times}} \\ & = \left( f(p) \right)^a. \end{align*}

How the completely multiplicative property works for prime powers

\[ f(mn) = f(m)f(n) \text{ whenever } (m, n) = 1. \]

\[ f(p_1^{\alpha_1} p_2^{\alpha_2} \dots p_k^{\alpha_k}) = f(p_1^{\alpha_1}) f(p_2^{\alpha_2}) \dots f(p_k^{\alpha_k}). \]

Euler totient function as Dirichlet product

\[ \varphi(n) = \sum_{d \mid n} \mu(d) \frac{n}{d} = \sum_{d \mid n} \mu(d) N\left(\frac{n}{d}\right) = (\mu * N)(n). \]

Proof of Theorem 2.18

Let $ f $ be multiplicative. We want to show that \[ \sum_{d \mid n} \mu(d) f(d) = \prod_{p \mid n} (1 - f(p)). \] Note the following: \[ g(n) = \sum_{d \mid n} \mu(d) f(d) = \sum_{d \mid n} (\mu f) (d) u\left( \frac{n}{d} \right) = (\mu f) * u. \] The functions $ \mu $ and $ f $ are multiplicative. Thus $ \mu f $ is multiplicative. Thus $ (\mu f) * u $ is multiplicative. Therefore \[ g(n) = g(p_1^{a_1} p_2^{a_2} \dots p_k^{a_k}) = g(p_1^{a_1}) g(p_2^{a_2}) \dots g(p_k^{a_k}). \] But \begin{align*} g(p_i^{a_i}) & = \sum_{d \mid p_i^{a_i}} \mu(d) f(d) \\ & = \mu(1) f(1) + \mu(p_i) f(p_i) + \underbrace{\mu(p_i^2) f(p_i^2) + \dots + \mu(p_i^{a_i}) f(p_i^{a_i})}_{=\,0} \\ & = 1 - f(p). \end{align*} From the two equations above, we get \begin{align*} g(n) & = g(p_1^{a_1}) g(p_2^{a_2}) \dots g(p_k^{a_k}) \\ & = (1 - f(p_1)) (1 - f(p_2)) \dots (1 - f(p_k)) \\ & = \prod_{p \mid n} (1 - f(p)). \end{align*}

§ 2.15: Formal power series

Product of formal power series

\begin{align*} A(x)B(x) & = \left( \sum_{n=0}^{\infty} a(n) x^n \right) \left( \sum_{n=0}^{\infty} b(n) x^n \right) \\ & = \left( a(0) + a(1)x + a(2)x^2 + \dots \right) \left( b(0) + b(1)x + b(2)x^2 + \dots \right) \\ & = a(0)b(0) + \Bigl( a(0)b(1) + a(1)b(0) \Bigr) x + \Bigl( a(0)b(2) + a(1)b(1) + a(2)b(0) \Bigr) x^2 + \dots \\ & = \sum_{k=0}^0 a(k)b(n - k) + \sum_{k=0}^1 a(k)b(1 - k)x + \sum_{k=0}^2 a(k)b(2 - k)x^2 + \dots \\ & = \sum_{n=0}^{\infty} \sum_{k=0}^n a(k)b(n - k). \end{align*}

Commutativity of product of formal power series

\[ A(x)B(x) = \sum_{n=0}^{\infty} \underbrace{\left\{ \sum_{k=0}^{n} a(k) b(n - k) \right\}}_{c(n)} x^n. \] \[ B(x)A(x) = \sum_{n=0}^{\infty} \underbrace{\left\{ \sum_{k=0}^{n} a(n - k) b(k) \right\}}_{c'(n)} x^n. \] \[ c(3) = a(0)b(3) + a(1)b(2) + a(2)b(1) + a(3)b(0). \] \[ c'(3) = a(3)b(0) + a(2)b(1) + a(1)b(2) + a(0)b(3). \]

Distributivity of multiplication over addition in formal power series

\[ A(x)\Bigl(B(x) + C(x)\Bigr) = A(x)B(x) + A(x)C(x). \] \[ \Bigl(B(x) + C(x)\Bigr)A(x) = B(x)A(x) + C(x)A(x). \] \begin{align*} A(x)\Bigl(B(x) + C(x)\Bigr) & = \left( \sum_{n=0}^{\infty} a(n) x^n \right) \left( \sum_{n=0}^{\infty} \Bigl( b(n) + c(n) \Bigr) x^n \right) \\ & = \sum_{n=0}^{\infty} \Bigl\{ \sum_{k=0}^{n} a(k) \Bigl( b(n - k) + c(n - k) \Bigr) \Bigr\} x^n. \end{align*} \[ A(x)B(x) + A(x)C(x) = \sum_{n=0}^{\infty} \sum_{k=0}^n a(k) b(n - k) x^n + \sum_{n=0}^{\infty} \sum_{k=0}^n a(k) c(n - k) x^n. \]

Determining coefficients of inverse of power series

\begin{align*} A(x)B(x) & = \sum_{n=0}^{\infty} \Bigl( \sum_{k=0}^{n} a(k) b(n - k) \Bigr) x^n \\ & = \Bigl( a(0) b(0) \Bigr) x^0 + \Bigl( a(0) b(1) + a(1) b(0) \Bigr) x^1 + \Bigl( a(0) b(2) + a(1) b(1) + a(2) b(0) \Bigr) x^2 + \dots \\ & = 1. \end{align*}

Inverse of geometric series

\begin{align*} A(x) & = 1 + ax + (ax)^2 + (ax)^3 + \dots, \\ B(x) & = 1 - ax. \end{align*} \begin{align*} A(x) B(x) & = \Bigl( 1 + ax + (ax)^2 + (ax)^3 + \dots \Bigr) (1 - ax) \\ & = \Bigl( 1 + ax + (ax)^2 + (ax)^3 + \dots \Bigr) - \Bigl( (ax) - (ax)^2 - (ax)^3 - \dots \Bigr) = 1. \end{align*}

§ 2.16: The Bell series of an arithmetical function

Bell series of $ f $ modulo $ p $

\[ f_p(x) = \sum_{n=0}^{\infty} f(p^n) x^n = f(1) + f(p) x + f(p^2) x^2 + f(p^3) x^3 + \dots \]

Theorem 2.24: Uniqueness theorem

\begin{align*} f(n) & = f(p_1^{a_1} p_2^{a_2} \dots p_k^{a_k}) = f(p_1^{a_1}) f(p_2^{a_2}) \dots f(p_k^{a_k}), \\ \\ g(n) & = g(p_1^{a_1} p_2^{a_2} \dots p_k^{a_k}) = g(p_1^{a_1}) g(p_2^{a_2}) \dots g(p_k^{a_k}). \\ \end{align*}

Example 1: Möbius function

\begin{align*} \mu_p(x) = \sum_{n=0}^{\infty} \mu(p^n) x^n & = \mu(1) + \mu(p) x + \mu(p^2) x^2 + \mu(p^3) x^3 + \dots \\ & = 1 - x + 0 + 0 + \dots \\ & = 1 - x. \end{align*}

§ 2.17: Bell series and Dirichlet multiplication

Power series multiplication

\[ A(x) = \sum_{n=0}^{\infty} a(n) x^n, \quad B(x) = \sum_{n=0}^{\infty} b(n) x^n, \quad A(x) B(x) = \sum_{n=0}^{\infty} \underbrace{\sum_{k=0}^n a(k) b(n - k)}_{c(n)} x^n. \]

Relationship with Dirichlet multiplication

\[ (f * g)_p(x) = f_p(x) g_p(x). \] \[ f_p(x) = \sum_{n=0}^{\infty} f(p^n) x^n, \quad g_p(x) = \sum_{n=0}^{\infty} g(p^n) x^n, \quad f_p(x) g_p(x) = \sum_{n=0}^{\infty} \sum_{k=0}^n f(p^k) g(p^{n-k}) x^n. \] \[ h = f * g = \sum_{d \mid n} f(d) g\left( \frac{n}{d} \right). \] \[ h_p(x) = \sum_{n=0}^{\infty} h(p^n) x^n = \sum_{n=0}^{\infty} \sum_{d \mid p^n} f(d) g\left( \frac{p^n}{d} \right) x^n = \sum_{n=0}^{\infty} \sum_{k=0}^{n} f(p^k) g(p^{n-k}) x^n. \]

Some steps of Example 1: \[ I(n)= \mu^2(n) * \lambda(n) \implies I_p(x) = \mu_p^2(x) \lambda_p(x) \] \[ I_p(x) = \mu_p^2(x) \lambda_p(x) \iff 1 = \mu_p^2(x) \cdot \frac{1}{1 + x} \iff \mu_p^2(x) = 1 + x. \]

Some steps of Example 2: \begin{align*} \frac{1}{1 - p^{\alpha}} \cdot \frac{1}{1 - x} & = \frac{1}{1 - x - p^{\alpha}x + p^{\alpha}x^2} \\ & = \frac{1}{1 - (1 + p^{\alpha})x + p^{\alpha}x^2} \\ & = \frac{1}{1 - \sigma_{\alpha}(p)x + p^{\alpha}x^2}. \end{align*} Note that $ \sigma_{\alpha}(n) = \sum_{d\,\mid\,n} d^{\alpha}, $ so \[ \sigma_{\alpha}(p) = \sum_{d\,\mid\,p} d^{\alpha} = 1^{\alpha} + p^{\alpha} = 1 + p^{\alpha}. \]

Some steps of Example 3: Showing That $ f(n) = 2^{\nu(n)} $ is multiplicative: \[ f(n) = 2^{\nu(n)}. \] \[ f(p_1^{\alpha_1} p_2^{\alpha_2} \dots p_k^{\alpha_k}) = 2^{\nu(p_1^{\alpha_1} p_2^{\alpha_2} \dots p_k^{\alpha_k})} = 2^k. \] \[ f(p_1^{\alpha_1}) f(p_2^{\alpha_2}) \dots f(p_k^{\alpha_k}) = 2^{\nu(p_1^{\alpha_1})} 2^{\nu(p_2^{\alpha_2})} \dots 2^{\nu(p_k^{\alpha_k})} = \underbrace{2 \cdot 2 \cdot \dots \cdot 2}_{k \text{ times}}. = 2^k. \]

§ 2.18: Derivatives of arithmetical functions

Ordinary derivative

\[ (f + g)' = f' + g'. \] \[ (fg)' = f'g + fg'. \] \[ \left( f^{-1} \right)' = \frac{-f'}{f^2} = -f' \cdot (f \cdot f)^{-1}. \]

Similarities in special derivative

\[ f'(n) = f(n) \log n. \] \[ (f + g)' = f' + g'. \] \[ (f * g)' = f' * g + f * g'. \] \[ \left( f^{-1} \right)' = -f' * (f * f)^{-1}. \]

Read on website | #mathematics | #number-theory | #book | #meetup

Notes on Introduction to Analytic Number Theory

2021-03-07T00:00:00Z

Notes on Introduction to Analytic Number Theory

This page contains an archive of notes from the book Introduction to Analytic Number Theory by Tom M. Apostol (1976).

Note that this set of notes is not meant to be a systematic exposition of analytic number theory. Instead this is just a collection of examples that illustrate some of the theorems in the reference textbook and intermediate steps that are not explicitly expressed in the book. These boards were used to aid the discussions during book discussion meetings. As a result, the content of these boards is informal in nature and is not intended to be a substitute for the book or the actual discussion meetings.

If you find any mistakes in the content of the board files, please create a new issue or send a pull request.

Notes on Chapter 2: Arithmetical Functions and Dirichlet Multiplication

More notes coming soon! We have all the meeting notes safely archived. Just need to format them and publish them here.

Read on website | #mathematics | #number-theory | #book | #meetup

Introduction to Analytic Number Theory Book Discussions

2021-03-05T00:00:00Z

Introduction to Analytic Number Theory Book Discussions

This meeting series is complete! There are no new meetings happening for this book. Go to the meetings main page to find out about current active meetings.

The following content on this page is an archive of the content as it appeared on the last day of meeting for this book.

Meeting time: 17:00 UTC from Tuesday to Friday, usually.^†

Meeting duration: 40 minutes.

Meeting link: bit.ly/spzoom2

Meeting log: 120 meetings

Reference Book: Introduction to Analytic Number Theory by Tom M. Apostol (1976)

Chapter notes: Notes

Started: 05 Mar 2021

Ended: 01 Oct 2021

† There are some exceptions to this schedule occasionally. Join our channel to receive schedule updates.

The primary reference book for these meetings is Introduction to Analytic Number Theory written by Tom M. Apostol. Admittedly, the book is quite expensive but you may find a relatively cheap paperback (softcover) copy on some websites.

These meetings are hosted by Susam and attended by some members of #math and #algorithms channels of Libera IRC network as well as by some members from Hacker News.

You are welcome to join these meetings anytime. If you are concerned that the meetings may not make sense if you join when we are in the middle of a chapter, please free to talk to us about it in the group channel. I can recommend the next best time to begin joining the meetings. Usually, it would be when we begin reading a new section or chapter that is fairly self-contained and does not depend a lot on material we have read previously.

Read on website | #mathematics | #number-theory | #book | #meetup

Susam's Meetup Pages

From Fill Prefix to TRAMP

Contents

Fill Prefix

Elisp Expressions in Replacement Strings

Keep Lines and Flush Lines

Keyboard Macros

DAbbrev

TAB vs M-i

Project Management

Eshell with TRAMP

Thanks

Notes on Mastering Emacs: Chapter 7: Conclusion

Contents

Third-Party Packages and Tools

Communities

Notes on Mastering Emacs: Chapter 6: The Practicals of Emacs

Contents

Exploring Emacs

Project Management

Xref

Xref Setup

Search Tools for Xref

Four Most Common Xref Commands

Xref and Dired

Working with Log Files

Highlighting

Auto-Revert Mode

Auto Revert Tail Mode

Browsing Tarballs

Dired: Thumbnail Image Browser

Preview Buffer Quirks

Working on Marked Files

Deleting Images

Tagging and Untagging

Using Tags

Display Buffer

DocView

DocView Resolution

TRAMP

Default Directory

Multi-Hops

EWW: Emacs Web Wowser

Invoking External Browser

Dired

Dired: Getting Started

Dired: Navigation

Dired: Marking and Unmarking

Dired: Operations

Dired: Copying or Renaming Between Buffers

Dired: More Keys

Dired-X

Dired: Working Across Directories

Shell Commands

Compiling in Emacs

Shells in Emacs

M-x shell

M-x ansi-term

M-x eshell

Notes on Mastering Emacs: Chapter 5: The Theory of Editing

Contents

Killing Text

Append Kill

Digit and Negative Arguments

Yanking Text

Yank Pop

Maximum Length of Kill Ring

Killing Lines

Transpose

Filling

Commenting

Search and Replace

Regular Expressions

Changing Case

Counting

Deleting and Keeping Lines

Splitting and Joining Lines

Examining and Fixing Whitespace Issues

Keyboard Macros

Text Expansion