Shell Eval

In this post, we will perform a few experiments to see the usefulness of the eval command for a particular scenario in a POSIX-compliant shell. At first, we prepare a test file that contains a space in its name and define a variable as follows:

$ echo lorem ipsum > "foo bar"
$ cmd='cat "foo bar"'

We will use this file and the variable in the experiments below. All output examples below are obtained using Dash 0.5.11 on a Debian GNU/Linux 11.2 (bullseye) system. Dash stands for Debian Almquist Shell which is a POSIX-compliant shell available in Debian. Any POSIX conforming shell should produce similar output. On Zsh, use the command emulate sh before running these examples to get similar output.

Experiment 1

Now simply enter $cmd as a command into the shell. The following error occurs:

$ $cmd
cat: '"foo': No such file or directory
cat: 'bar"': No such file or directory

The error occurs because the above command expands to the command cat followed by two arguments: "foo and bar". Such an expansion occurs due to a concept known as field splitting. Quoting from section 2.6.5 of POSIX.1-2017:

After parameter expansion, command substitution, and arithmetic expansion, the shell shall scan the results of expansions and substitutions that did not occur in double-quotes for field splitting and multiple fields can result.

The shell shall treat each character of the IFS as a delimiter and use the delimiters as field terminators to split the results of parameter expansion, command substitution, and arithmetic expansion into fields.

By default, the space character belongs to IFS. Here is an example command to verify this:

$ printf "$IFS" | od -tcx1
0000000      \t  \n
         20  09  0a
0000003

The hexadecimal code 20 in the output confirms that the space character is present in the value of IFS. Therefore, according to the POSIX specification, $cmd first expands to cat "foo bar", then it is split into three fields cat, "foo and bar" and then the command cat is executed with two arguments "foo and bar". Since no files with those names exist, an error occurs.

Experiment 2

Next we try to double-quote the previous command to prevent field splitting and see what happens:

$ "$cmd"
dash: 8: cat "foo bar": not found

The excerpt from the POSIX.1-2017 specification quoted in the previous section shows that field splitting does not occur for variable expansions within double quotes. So the entire expansion cat "foo bar" remains intact as a single field and is then executed as a command. Since there is no such weirdly named command, we get the above error.

Experiment 3

Field splitting leads to an error as seen in the first experiment. Preventing field splitting by double-quoting the variable expansion also leads to an error as seen in the second experiment. How do we execute the command in the cmd variable?

We need a way to somehow get the shell to parse cat "foo bar" like a shell normally does, i.e. treat each unquoted token as a separate field and each quoted token as a single one. How do we get the shell to do that? Well, we can just invoke the shell itself to parse our command:

$ sh -c "$cmd"
lorem ipsum

But the above command invokes a new shell process. Can we avoid that? Yes, using the eval command:

$ eval "$cmd"
lorem ipsum