Written by Chrisitan Lee Seibold. September 2-4, 2020. Updated with more information on Multics, and adapted for Gemini, on September 25, 2023.
A Shell is a textual interface that allows you to type commands to instruct the computer in doing something, including running programs and running files that list a sequence of commands, initially called “runcoms” and, later, shell scripts. Shells seem to stem from one operating system, MIT’s CTSS, which was very influential in both the Unix line, and the (CP/M) DOS/Windows line of Operating Systems. As we will see, MIT, DEC, and Bell Labs influenced the computer space deeply. In this article, we discuss the Unix line of shells, starting with RUNCOM and Multics, a Project MAC Operating System with the goal of being a successor to CTSS.
Unix Shells have had a very long history, and it all starts with a program written by Louis Pouzin for the MIT CTSS Operating System, called RUNCOM (which stood for “run commands”). It executed commands from a file, called “a runcom”. According to Kernighan and Ritchie[1], “rc” configuration files from Unix descended from this. Tom Van Vleck also gives origins of Unix’s use of “rc” to RUNCOM [2], and notes that the first time he read the term “shell” was from Multics documentation created by Doug Eastwood. According to Louis Pouzin, he coined the word “shell”. [3]
Multics started development in 1964 as a project of MIT’s Project MAC, in collaboration with GE and Bell Labs. During the year of 1964, Christopher Strachey visited MIT. Louis Pouzin thought Strachey’s macro-generator design, particularly the “techniques for quoting and passing arguments”, was a good base for a command language. Pouzin wrote a paper and a flowchart designing what would become Multics’ “shell” (a term he coined) just before he left. After leaving, this shell was implemented into Multics by Glenda Schroeder and another programmer from GE. [3]
This early implementation of the shell included the basic syntax of `command arg1 arg2` with arguments separated by spaces and terminated by semicolons. It also included strings as return values, command substitution, and iteration. Iteration allowed you to write multiple elements in parentheses so the shell would run each version of the command with each element. So, `print (a b c).epl` became `print a.epl; print b.epl; print c.epl` . The shell also allowed you to substitute in the results of a command in different ways, where `[cmd]` would substitute the returned string into the command (called an active command), `|[cmd]` would substitute the returned string into the command line as another command (called a neutral command), and `||[cmd]` would not substitute anything into the command (called an empty command). [4]
Pipe syntax was later added to Multics in the 80s to make its already-existing IO redirection features more convenient. Before this addition, according to Vleck, `io_call` and stream manipulation could achieve the same effect as Unix’s pipes, just without the convenient pipe syntax. [5] However, according to Dennis Ritchie, he doesn’t “think this is true, or is true only in a weak sense” because “Multics spliceable IO modules required that the modules be specially coded in such a way that they could be used for no other purpose”. However, he does note that it did have a “general IO redirection mechanism … embodying the name IO streams”, but that the notation was “very clumsy”. [6] It should be noted that the Unix developers left the Multics project well before Multics' release date and well before many of the significant changes to Multics that happened.
In 1969, the Multics Condensed Guide, which covered the "commands available in Limited Initial Multics (LIM)", it included among the features of Multics' Command Processor iteration, active commands, literal strings, pattern matching (`*`, `**`, `=`, and `==`), the `who` and `echo` commands, and the `runcom` command.[20] The `runcom` command is described as follows:
Purpose: To permit the user's next input lines
to be taken from the ascii text segment, specified by path, rather than
from the console. arg1 ... argn are
optional arguments to be inserted into
the text of path.
Notes: Each argument is inserted into the text
of path as indicated by the include (&)
sign, followed by a decimal number,
where:
&1 is the first argument,
&2 is the second argument, etc.
As shown above, the `runcom` command ran script files containing commands, and also had the ability to insert arguments into the script via an "include (&) sign." [20]
Multics also had a search path that was used to search for commands that could be used within the shell. In 1973, the Multics Command Processor notably included active functions, exec_com segments (aka. files) ending in the `.ec` suffix, a mail and message facility, the `who` command, the `help` command, and a modifiable search criteria where commands would be ran from. [19]
In at least 1979, it additionally had argument substitution and control flow (including `&goto` and `&if`, among others) within exec_coms, a start_up.ec exec_com that would run upon login, and fairly advanced pattern matching (`*`, `**`, `?`, `=`, `==`, `===`, and `%`). [18]
Due to dissatisfaction with the Multics system, Bell Labs withdrew from the project In 1969. [17] Plans and development for a new Operating System for the PDP-7 were started in 1969. Much of this work, including the command interpreter (shell), was done by Ken Thompson. This shell eventually had IO redirection, inspired by Multics’ IO streams. In 1970, Bell Labs purchased a PDP-11, and work on a port of the Operating system to the PDP-11 was started. 1970 was also the year the name “Unix” was proposed. Later, in 1972, pipes, a “specific form of coroutine”, were added to the shell, by a suggestion from M. D. McIlroy [6], who invented the concept years prior, in 1964 (p. 68) [17]. While this was not known at the time, Dartmouth Time-Sharing System (DTSS) also had a very similar concept to pipes. At first, pipes were used by stacking up the commands one after another. However, later on, the syntax was switched so that the `>` character was used. [6] According to both John Mashey[8] and David Korn[9], this shell also had `goto` and `if` commands.
While Multics originally had the search path, Unix decided to give up this idea initially. According to Doug McIlroy, the search path was added in v3 of Unix. [18] However, according to Brian Kernighan, the search path was added in PWB Shell. (p. 132) [17]
Work on a replacement shell for Unix was started in mid-1975, according to Stephen Bourne. [7] This shell was written by John Mashey and a group of other people, including Alan Glasser, and was distributed as part of the Programmer’s Workbench UNIX. New commands and features were added to Unix’s shell, including `switch` , `while` , and variables, 3 of which were derived from per-process data. Some of the existing commands were also improved, including `if/else/endif` . [8] PWB Shell also included a search path so that a sequence of directories could be searched for commands, which was later copied by Bourne Shell. [17]
In 1975, it was decided that the shell for Unix should be rewritten to fix issues. At this time, Ken Thompson went off to Berkeley for a year, so the shell was mainly written by Stephen Bourne at Bell Labs. The first version of the shell was deployed in 1976. [7]
Much of the Bourne Shell’s syntax was inspired by Algol68. The shell added shell scripts, multi-character variables, here docs, command substitution, path searching, interruptible wait, pattern matching, and string quoting. There were no length restrictions on strings, goto was removed, there were no comments, environment variables were added later, in 1978 [7], and functions, `echo` , and `pwd` were added in 1982, as part of Unix System V [11]. Bill Joy suggested to Bourne that Job Control and History should be added to the shell, to which Bourne disagreed. [7]
Unix version 7, with Bourne Shell as default, was released in 1979.
C Shell was started in 1978 by Bill Joy at Berkeley. According to David Korn, C shell introduced command history and an editing functionality. [9] In the intro to The UNIX C Shell Field Guide, Bill Joy mentions the shell’s original ideas were the history mechanism, which was inspired by “the history and DWIM features of Interlisp” (invented by Warren Tietelman), aliasing, which was patterned “roughly on Lisp reader macros”, and job control, which was “added … by Jim Kulp”. [10]
C shell also had `&` for background commands, here docs, CDPath, directory stacks, path hashing, brace expansion (aka. alternation - used in pattern matching), expression evaluation, and command groups. On page 73, the field guide mentions pathname variables, which allowed you to pick specific parts of a path stored inside a variable using the syntax: `$p:x` , where `x` can be `r` for root, `h` for header, `t` for tail, or `e` for extension.
On page 41 of the Korn Shell manual, it is mentioned that Tilde Notation came from C Shell. [11]
Korn Shell was developed by David Korn at Bell Labs and was released in 1983. David Korn created the precursor to the Korn Shell as a “form interpreter” by modifying the Bourne Shell, allowing built-in commands to be used in I/O redirection, and adding the `echo` , `pwd` , and `test` built-in commands. [9]
David Korn later implemented the first version of Korn Shell prior to the UNIX System V shell. This first version took history, aliases, and job control from C Shell. [9] According to the Korn Shell Manual, tilde notation, Job Control, directory stack, and the `logout` command all came from C shell (pages 41, 97, 273, and 267 respectively). [11]
Korn Shell also had vi line editing mode, written by Pat Sullivan, and emacs line editing mode, written by Mike Veach. [9]
The 1988 version of Korn Shell extended pattern matching to be similar to regexes found in sed and grep at the time. This version was used as the basis for the POSIX.2 standard in 1992, aka. “IEEE POSIX 1003.2” and “ISO/IEC 9945-2”, which created POSIX standards for shells and utilities. [9]
The Korn Shell Manual [11], from 1992, also mentions the following features:
TENEX C Shell was developed by Ken Greer. Tcsh was based on C Shell, but added additional features, including “command and filename recognition and completion” written by Mike Ellis, and inspired by the TENEX OS by BBN. [12]
Kenneth Almquist first released Almquist Shell in 1989. It was a reimplementation of the System V shell with added features, including:
The Bourne-Again shell was developed by Brian Fox for the GNU Project. It was first released in 1989. Chet Ramey took over development in 1994. It was based on the Bourne Shell. It had brace expansion (from C Shell), programmable command line completion, command line editing, command history, the directory stack, functions, arrays, integer arithmetic via the `(())` syntax, and POSIX command substitution via the `$()` syntax.
In 2004, bash introduced associative arrays.
RC was the Shell for Plan 9 and Unix version 10, and was developed by Tom Duff. It replaced the Algol68 like control structures of Bourne with C-like ones. Variables are a list of strings instead of one string, where the list is created by splitting arguments by spaces. This allowed you to reference a specific argument within a variable. The exit status of commands was a character string describing the error. Curly brackets were used for a sequence of commands.
Z Shell was created by Paul Falstad and released in 1990. It had editing of multi-line commands, spelling completion/auto-correction and auto-fill of command names, and themeable prompts. [14] It also had termcap support and login/logout watching. [15]
It now has various compatibility modes, loadable modules, and named directories.
The POSIX.2, Shell and Utilities, Command Interpreter (IEEE Std. 1003.2-1992) standard specified what a POSIX-compliant shell should include. These specifications were based on the System V Shell (Bourne Shell) , “with enhancements from Korn Shell”. [16]
Debian Almquist Shell was a port of Almquist Shell from NetBSD to Debian Linux by Herbert Xu, in 1997. Its goals were POSIX conformance and slim implementation. Internationalization, localization, and multi-byte character encodings, however, were not implemented. It also had optional history and GNU readline line editing support.
Ubuntu adopted dash in 2006. Debian adopted it in version 6 (Debian Squeeze). Although both Ubuntu and Debian adopted dash, bash still remains the default login shell for interactive use.
Today, dash, bash, and Bourne are the shells still in use in the Linux and macOS world. Mksh is a descendant of Korn Shell, and is used on Android. FreeBSD uses tcsh. NetBSD uses ash. And finally, OpenBSD uses ksh. All the shells above have had an immense influence in the computer world, even crossing over to Windows. However, there is another story. This other story also involves CTSS. However, it quickly diverges, and, in the end, merges into one story. This story is the story of where CP/M, DOS, and Windows’ command line Shells came from. This is the story that will be covered in the next article.
Note: If you find any errors within this article, please email them to me at: christian.seibold32@outlook.com
This is a list of shell features and which shell implemented it first:
[1] https://kb.iu.edu/d/abnd
[2] Unix and Multics, Tom Van Vleck: https://www.multicians.org/unix.html
[3] The Origin of the Shell, Louis Pouzin: https://multicians.org/shell.html
[4] MULTICS SYSTEM-PROGRAMMERS' MANUAL Section BX.1.00 - Multics Command Language (1968): https://multicians.org/mspm-bx-1-00.html
[5] Glossary of Multics acronyms and terms, Tom Van Vleck: https://multicians.org/mgp.html#pipes
[6] The Evolution of the Unix Time-sharing System, Dennis Ritchie: https://www.bell-labs.com/usr/dmr/www/hist.html
[7] Early days of Unix and design of sh, Stephen Bourne: https://www.youtube.com/watch?v=2kEJoWfobpA
[8] Mashey Shell Repost, John Mashey: https://www.in-ulm.de/~mascheck/bourne/n.u-w.mashey.html
[9] ksh - An Extensible High Level Language, David Korn: https://www.in-ulm.de/~mascheck/bourne/korn.html
[10] The UNIX C Shell Field Guide, Gail and Paul Anderson: https://archive.org/details/unixcshellfieldg00ande/page/326/mode/2up
[11] The Korn Shell User and Programming Manual, Anatole Olczak: https://archive.org/details/kornshelluserpro1992olcz/page/368/mode/2up
[12] https://groups.google.com/g/net.sources/c/BC0V7oosT8k/m/MKNdzEG_c3AJ?pli=1
[13] A reimplementation of the System V shell: https://groups.google.com/g/comp.sources.unix/c/A6cnyKX-Gq4/discussion
[14] zsh - a ksh/tcsh-like shell (part 1 of 8): https://groups.google.com/g/alt.sources/c/tVgN49u8Ax4/m/7VgQlHZ4bJMJ
[15] zsh - a ksh/tcsh-like shell (part 2 of 8): https://groups.google.com/g/alt.sources/c/NiVcvSh3P04/m/tY4Zswxb0UsJ
[16] Jesperson, Hal. "Emerging standards." UNIX Review, vol. 11, no. 3, Mar. 1993, p. 30+. Gale Academic OneFile, https://link-gale-com.briarcliff.idm.oclc.org/apps/doc/A13761630/AONE?u=briarcliffu&sid=AONE&xid=575043a6. Accessed 31 Aug. 2020.
[17] Brian W. Kernighan. UNIX: A History and Memoir. Kindle Direct Publishing, 2020.
[18] Honeywell's New User' Introduction to Multics - Part I and Part II (CH24-0 and CH25-0, Nov. 1979)
http://bitsavers.trailing-edge.com/pdf/honeywell/large_systems/multics/CH24-0_multicsIntrPt1_Nov79.pdf
http://bitsavers.trailing-edge.com/pdf/honeywell/large_systems/multics/CH25-0_multicsIntrPt2_Nov79.pdf
[19] Users' Guide: Basic Introduction to Multics, Intended as a Guide to New Users. MR1.0 (AL40, Nov. 1973) http://bitsavers.trailing-edge.com/pdf/honeywell/large_systems/multics/swenson/al40-00.731100.mr1-0.users-guide.82.pdf
[20] Multics Condensed Guide, Revision 2 (June 1969) http://bitsavers.trailing-edge.com/pdf/honeywell/large_systems/multics/swenson/6906.multics-condensed-guide.pdf