Getting Started with Terminal and a Shell

Getting Started with Terminal and a Shell

You can think of Terminal's command-line interface as a way to look "under the hood" of Mac OS X. You can see what's going on, and even poke around a bit, if you like. However, it's important to realize that working at the command line isn't really working in Terminal so much as through it. Terminal is simply a tool that acts as an intermediary between you and the command-line environment; it doesn't do much of anything itself (somewhat like the steering wheel and pedals of a car, which don't actually move the car down the road, but enable you to direct the engine and wheels).

The most important program in a command-line interface is the command shell. The shell is your agent, relaying your commands to the kernel and other programs, showing you their responses, and providing a programming environment in which you can execute commands and scripts. Modern shells also provide a great deal of assistance in managing the command line—to help make using other command-line programs more efficient. The default shell in OS X is called tcsh, and is used whenever you launch Terminal (unless you've specifically told the system to use a different shell); we'll look at tcsh later in the chapter. More advanced tcsh programming, including looping and if/else processing, is beyond the scope of this book, but if you'd like to read further, the tcsh manual page (and the Internet) contain a wealth of further information on shell programming.

A command line generally consists of a command program (the first word of the line) followed by arguments that tell the program exactly what to do. When you press the return key, the shell interprets the command line, passes it to the specified program, and then displays the program's output, if there is any.

Most commands are individual programs, each much smaller than a full Mac application and located in one of the directories you saw in your PATH. However, each shell contains a set of built-in commands as well. In tcsh, typing builtins <RETURN> (try it now) lists these internal commands, and you can see instructions for using them in the tcsh manual page (under "Builtin commands"; I'll talk more about manual pages in a bit). Builtins like set and print-env manage tcsh itself; builtins like foreach and while provide programming functionality, and builtins like ls-F and time provide frequently used capabilities without execution of external programs.

Command Structure: An Example

Let's try a few simple commands and analyze what they do. The meanings of each argument are determined by the command they follow, but most arguments specify either files to manipulate (read, write, search, move, etc.) or options that modify the program's behavior.

Options (also called flags) modify the behavior of a command; for example, the grep command's -i option makes grep (a text-searching utility) ignore case when searching (the default is to search case sensitively). To help distinguish options from other arguments, options are normally preceded by a dash (-). They can be combined in any order; however, they must precede other (non-option) arguments on the command line. In addition to on/off switches, more complicated options may be followed by additional text, as a sort of argument to the option itself.

Warning 

Unix filesystems and Unix programs are case sensitive, whereas the Classic Mac OS, OS X's Finder, and Apple's HFS and HFS Plus file systems are case insensitive and case preserving. For example, using UFS (the Unix File System), you can save different files named MyFile and myfile in the same directory, but on HFS Plus volumes the names are equivalent, so the second file overwrites the first. This is also important because Unix programs tend to be case sensitive, and assume that the file system is as well. For example, if you search for myfile in Terminal (even on an HFS Plus volume) the program you're using (locate, tcsh, ls, grep,etc.) will ignore MyFile, as it is assumed to be a different file. (Although you can force tcsh's command completion to be case insensitive with set complete=enhance <RETURN>. See the tcsh manual page under "Completion and listing" for full details.) For more information on the challenges of combining Unix and the Mac OS, check out http://www.usenix.org/publications/library/proceedings/usenix2000/invitedtalks/sanchez_html/.

Finally, although the order of options is unimportant, the order of other arguments is generally very important. For example, the command grep/bin/sh/etc/rc would search for the string /bin/sh in the file /etc/rc (and would find it); if you instead used grep/etc/rc/bin/sh, it would search for any occurrence of the string /etc/rc in the file /bin/sh (and wouldn't find it).

Table 15.1 lists a few commonly used commands, along with their basic functions, some possible options, and normal arguments for each (if applicable).

Table 15.1: Some Basic Commands

Command

Function

Options

Other Arguments

ls

List files

"-l": use long listing format

"-F": show file-type suffixes

Which files to list, or which directories' contents to list (defaults to current directory)

pwd

Print Working Directory

N/A

N/A

cd

Change Directory

N/A

Which directory to change to (defaults to user's home directory)

grep

Search

"-i": search case insensitively

"-v": search for non-matching lines

"-l": list files that match, but don't show the actual matches

1: string for which to search

2: file(s) to search

curl

Download or Upload files

"-i": include http header in output

"-o file": save output in file (otherwise it is sent to standard output)

"-O": use the remote filename (from the URL) as the local filename

"-u user:password": provide this username and password to the remote sever; if pass word isn't supplied, curl will prompt for it (this is more secure, as command-line arguments may show up in ps and command history)

URLs (HTTP, FTP, or various other

formats) to get or put

open

Open in Finder

"-a": specify application for opening file

Which file/folder/application to open

Basic Commands: A Walk-through

It's time to see some of the commands I just talked about in action. Launch Terminal (or open a new Terminal window if it's already running) and follow along.

  1. Type ls <RETURN>. The output of the command lists the contents of the default directory. But what directory is that?

  2. Type pwd <RETURN> to see what the current working directory is. (The working directory is the one you are currently working in; it's a bit like the frontmost window in the Finder.) Chances are it will be /Users/YourUsername, your home directory—this is the default directory when you open a new shell session (Terminal window).

  3. Type ls / <RETURN>. This command shows the contents of the top level of the startup disk (called the root directory—distinct from the root user). If you had typed just ls <RETURN>, you would have gotten a listing of the current working directory—in this case your home folder. However, by providing a directory path (in this case, /) as an argument, you instructed the ls command to give you a listing of that directory, instead.

    Warning 

    In the Classic Mac OS (and from within applications in the Classic Environment), the top level of the file system is the Desktop, and each mounted volume appears there, along with any files in the Desktop Folder(s) of any mounted disks. In the OS X Finder, the top level of the file system is Computer, and all mounted disks and a Network item are there. In Unix (and therefore in Terminal in OS X), the top level of the file system is called "/" (the "root" of the file system) but is only the top-level directory of the startup volume; other disks and removable volumes are mounted inside /Volumes. (These are the most common layouts, although there are others that crop up in specific circumstances, such as Internet Explorer's Address field.) Because of these different—and confusing—approaches, it's important to realize that some items (particularly disks) are in different places depending on whether you're in a Classic, Cocoa, Carbon, or Unix application.

  4. To move to the root directory type cd / <RETURN>. Note that after this command, the prompt shows a slash instead of a tilde, since you're no longer in your home directory.

    Tip 

    As I discussed in Chapter 1, ~/ is a way of referring to your home directory, and / is a way of referring to the root directory. In the shell (Terminal), you can use ~ or ~/ to refer to your home directory. For example, to change to your home directory, type cd ~ <RETURN> or cd~/ <RETURN>. To refer to another user's home directory, use the username as well, as in ~username or ~username/ (or ~root or ~root/ for the home directory of the root user, /var/root by default).

  5. Now that the root level of your startup disk is the working directory, using ls without any arguments lists the files there, instead of listing those in your home directory. You can add the -l option (the letter "l", not the number one) for a detailed ("long") listing and the -F option to view file-type suffixes. (Since options can be combined and reordered, ls -lF, ls -Fl, ls -l -F, and ls -F -l are all equivalent.) In the detailed listing you'll see a few items with a dash in the first column and no suffix; these are standard files. Entries with a "d" in the first column and a slash suffix are directories. Files with an "l" in the first column and an @ symbol at the end are symbolic links (the Unix version of OS X's aliases); the long listing indicates the destination/target of the link using an arrow (e.g., var@ -> private/var).

  6. Type pwd <RETURN> to confirm that you're actually at the root level of the startup disk.

  7. Type cd <RETURN>. Note the prompt—it looks like you're back in your home directory, doesn't it? Type pwd <RETURN> to be sure. When you use the cd command without any arguments, it automatically (and conveniently) changes the active directory to your home directory.

Tip 

Files and directories in Unix normally have lowercase names and no spaces (spaces would complicate commands); for example, etc and usr. Apple has traditionally capitalized the first letter of each word in file and folder names and (in OS 9 and earlier) often used spaces (e.g., System Folder, Control Panels). In OS X, Apple avoids spaces in filenames, but does use capital letters on Apple-installed items (e.g., StartupItems). This makes it easy to figure out that /etc/rc is a traditional Unix file, while /System/Library is an Apple creation. However, in Terminal, spaces in file or path names can create problems, because they can be interpreted as spaces between command arguments. If you need to type a filename with a space or punctuation character, you should either put quotes around the whole filename or pathname (e.g., ls '/SystemFolder' <RETURN>), or use a backslash (\) just before the special character to escape it, so it's interpreted as a standard ASCII character, rather than its special meaning (e.g., ls /System\ Folder <RETURN>).

Learning New Programs: The man Command

When you need to use an unfamiliar command-line program, the man command is your friend. The man program takes one or more program names as arguments, and shows a descriptive manual page (often called the man page) for each listed program. As an example, typing man ls <RETURN> describes the ls program, including all 26 of its options. Most of these options are rarely needed, but you'll find several that can be very useful, as seen in Table 15.2.

Table 15.2: Some Useful Options for ls

option

function

-F

Include file-type suffixes

-R

List subdirectories recursively

-a

Show all files (normally, ls and other commands ignore files with names that start with a period, often called dotfiles; you'll remember from Chapter 5 that these files are normally invisible in the Finder)

-d

Show directories and symbolic links, instead of the files they contain or point to

-l

Use long format

-k

Show sizes in kilobytes instead of bytes

-t

Sort by modification date, recently modified items first

Most programs have fewer options than ls, and most options are never needed. However, as a new Unix user, you're likely to find yourself looking up many new and seldom-used commands for a while. Fortunately, man is powerful and easy to use. Unfortunately, manual pages are not always well written, and early versions of Mac OS X shipped with obsolete and incomplete man pages; Apple improved the state of man pages with OS X 10.1, and man pages in 10.2 are now generally correct and current.

Tip 

To learn more about the man command itself, try man man. To find help when you aren't sure what command you need, try man -k word, where word is a word related to what you want to do; this will search the installed manual pages for occurrences of word in command names or descriptions. (You can also use apropos word, which is equivalent to man -k word.)

Finally, if reading long manual pages in Terminal is not your cup of tea, there are several utilities that present manual pages using a nice graphical interface; many even include clickable links to the manual pages of related commands. To view the current batch of these utilities, do a search for "man page" at on VersionTracker.com or MacUpdate.com.

Shells: sh versus csh

Steve Bourne was the creator of the first Unix shell, and with classic Unix terseness named it sh. The sh shell is extremely powerful, but, in a deliberate attempt to make interactive use as similar as possible to programming or "scripting," lacks many convenience features. Several alternative shells are included in OS X, notably csh (the incompatible C shell, with syntax closer to the C programming language), Bourne's own sh-compatible bash (the "Bourne Again Shell"—the default on Linux systems), and the csh-compatible tcsh—OS X's default.

As a rule, shells are fully compatible with either sh or csh, layering additional features on top of one or the other. For example, OS X uses a copy of bash as sh, and tcsh as csh (see Listing 15.1). This works because bash is backwards compatible with sh, and tcsh is compatible with csh (more details are available in their man pages). For maximum portability, system scripts are normally written in the basic sh syntax, and avoid advanced features of other shells that may be unavailable on some systems. In fact, the Unix boot process is based on sh scripts. This makes understanding and modifying the system startup process much easier, as the basic levels of the system consistently use the sh dialect.

Listing 15.1: OS X includes several shells; here we see that sh and tcsh are located in /bin. If we look for other shells in /bin, we find that there are actually five. Finally, we see that sh is really bash, and that csh is actually the same as tcsh.
Start example
[g4:~] power% which sh tcsh
/bin/sh
/bin/tcsh
g4:~] power% ls -l /bin/*sh
-rwxr-xr-x  1 root  wheel  540884 Nov 22 19:47 /bin/bash
-r-xr-xr-x  1 root  wheel  315136 Nov 22 19:47 /bin/csh
-r-xr-xr-x  1 root  wheel  540884 Nov 22 19:47 /bin/sh
-r-xr-xr-x  1 root  wheel  315136 Nov 22 19:47 /bin/tcsh
-rwxr-xr-x  1 root  wheel  828780 Nov 22 19:47 /bin/zsh
[g4:~] power% diff --report-identical-files /bin/sh /bin/bash
Files /bin/sh and /bin/bash are identical
[g4:~] power% diff --report-identical-files /bin/tcsh /bin/csh
Files /bin/tcsh and /bin/csh are identical
End example
Note 

While there are many differences between csh and sh syntax, they do not affect basic command invocation; rather, they show up in more advanced areas, such as variables, error redirection, conditional execution, and looping. Most examples in this chapter will work in any of the shells provided with OS X, but system scripts are written for sh, and user commands are normally written for tcsh. For more on variables in tcsh (which are handled differently than in sh-based shells), see the "Variables" section later in this chapter.

As mentioned above, the default shell for OS X is tcsh . This is a good choice, as tcsh is sophisticated and has many convenient features that make using a command line easier and more efficient. Several other shells are provided with OS X, including bash, that provide many of the same conveniences.

Using Another Shell

To try another shell temporarily, just type its name within your current shell (e.g., bash<RETURN>). This will run the specified shell as a command (i.e., within your default shell), until you exit (by typing exit <RETURN> or logout <RETURN>, or pressing control+D), at which point you'll return to tcsh.

Command-Line Assistance

One of the areas in which tcsh excels is assisting you with interactive use. The tcsh shell can complete words automatically for you, offers good support for editing of command lines, and stores previous command lines for reuse. In addition, it has a spell checker that is surprisingly successful at DWIM (do what I mean). Table 15.3 includes some keyboard shortcuts for command-line editing in tcsh (most also apply to bash).

Table 15.3: Keyboard Shortcuts in the tcsh shell and bash

Keystroke

Effect

control+U

Clear line (this works in most Unix programs, and will either clear the whole line, or clear only the part of the line from the cursor to the beginning, leaving everything afterwards)

control+A

Move cursor to beginning of line

control+E

Move cursor to end of line

tab

Auto-complete current command/file/directory name (a beep here means there are ambiguous possibilities—type a bit more to remove the ambiguity, and then press tab again)

up arrow

Show previous command in history

down arrow

Show next command in history

Variables

Programming languages use variables to store working information, and shells are no exception. The tcsh shell uses both local and environment variables. Local variables are internal to a running copy of the shell, whereas environment variables are passed along to other programs the shell executes. Listing 15.2 shows an example of local variables in action.

Listing 15.2: Local variables in tcsh
Start example
[g4:~] power% set
PROXYFTP        ()
PROXYHTTP       ()
_       cd ..

addsuffix
argv    ()
cwd     /Users/power
dirstack        /Users/power
echo_style      bsd
edit
gid     20
group   staff
history 100
home    /Users/power
loginsh
owd     /Users/power/fink-0.5.0a-full
path    (/sw/bin /sw/sbin /bin /sbin /usr/bin /usr/sbin /usr/X11R6/bin)
prompt  [%m:%c3] %n%#
prompt2 %R?
prompt3 CORRECT>%R (y|n|e|a)?
promptchars     %#
shell   /bin/tcsh
shlvl   1
status  0
tcsh    6.10.00
term    vt100
tty     ttyp4
uid     504
user    power
version tcsh 6.10.00 (Astron) 2000-11-19 (powerpc-apple-darwin)
options 8b,nls,dl,al,sm,rh,color
End example

If you look at what tcsh keeps in its local variables, you should recognize a lot of shell housekeeping information, such as the prompt string, working directory, username and user ID (uid), and various other bits. These variables are set automatically during tcsh's initialization process, but tcsh's behavior can be customized by setting a variety of additional special variables (explained under "Special shell variables" in the tcsh man page).

To manage local variables, tcsh provides the set command. Without any arguments, set shows all local variables and their values (see Listing 15.2). To set a variable, use the command set myvariable=myvalue <RETURN>, which creates myvariable (if it wasn't already defined) and sets its value to myvalue. For example, the following command creates a new variable, quantity and sets its value to 10:

   [g4:~] power% set quantity=10
   [g4:~] power%

The echo command prints the arguments it receives from the shell (which does variable expansion before passing them along to echo), so it is often used to show the values of variables. For example:

   [g4:~] power% echo quantity
   quantity
    [g4:~] power%

Not what you expected? The above command told the shell to simply echo the word quantity. To use the value of a variable in a command line, precede its name with a dollar sign ($):

   [g4:~] power% echo $quantity
   10
   [g4:~] power%

This usage tells the shell to send the value of the quantity variable to the echo program.

Note 

Unfortunately, sh and csh use different syntax to set variables, but use them similarly once set. More detail is available under "Variable substitution" in the tcsh manual page and under "Variables and Parameters" and the export command description in the sh manual page.

Environment variables are very similar to local variables, but as I mentioned previously, environment variables are passed along to other programs the shell executes. Environment variables are more important than local shell variables because they affect every program you run (local variables only affect the shell itself). To help you distinguish them from local variables, environment variables generally use capitalized names (local variables are usually low-ercase). The tcsh shell complicates things a bit by using several local variables (such as path) that correspond to environment variables (such as PATH), but for the most part the distinction holds true. Listing 15.3 shows environment variables used by tcsh.

Listing 15.3: Environment variables in tcsh
Start example
[g4:~] power% setenv
TERM=vt100
USER=power
HOME=/Users/power
SHELL=/bin/tcsh
HOSTTYPE=macintosh
VENDOR=apple
OSTYPE=darwin
MACHTYPE=powerpc
SHLVL=1
PWD=/Users/power
LOGNAME=power
GROUP=staff
HOST=hostname.example.com
PATH=/sw/bin:/sw/sbin:/bin:/sbin:/usr/bin:/usr/sbin:/usr/X11R6/bin
MANPATH=/sw/share/man:/sw/man:/usr/local/share/man:
/usr/local/man:/usr/share/man:/usr/X11R6/man
INFOPATH=/sw/share/info:/sw/info:/usr/local/share/info:
/usr/local/lib/info:/usr/local/info:/usr/share/info
PERL5LIB=/sw/lib/perl5
[g4:~] power% setenv HOSTTYPE g4
[g4:~] power% echo $HOSTTYPE
g4
[g4:~] power%
End example

The most important variable for everyday use is PATH, which controls where the shell (and other programs) looks for commands. Think how tiresome typing /usr/bin/ls /Users/username <RETURN> each time you wanted to use the ls command would become! With the PATH environment variable set to include the directory that contains the ls program (/usr/bin) and the PWD variable set to /Users/username, tcsh can expand ls to /usr/bin/ls for you, and ls knows to display /Users/username if you don't specify otherwise.

To define and assign environment variables, use the setenv command; its syntax is similar to that of set, but with a space instead of the equal sign.

Command Aliases

In addition to programs and builtins, a shell command might also be an alias. Not to be confused with Finder aliases, command aliases are user-defined commands that can be used to override existing commands (perhaps by making rm require confirmation before deleting files) or to create new shortcut commands with your favorite options, such as making l mean ls -lF. An alias command consists of a command word (the alias) and the command (and any options) you want executed when you use the alias (called the expansion).

To create a new alias, you need to use the alias command with two or more arguments: the first argument is the name of the new alias, and the rest of the line its expansion. For example, typing alias shedit pico ~/.cshrc <RETURN> creates a new alias, shedit, that, when typed in Terminal, is expanded by the shell to the command pico ~/.cshrc (which opens your shell configuration file in the pico text editor). I created this alias for myself because I could never remember the name ".cshrc"—I would always type ".schrc" or some other variation. Now whenever I want to edit my shell configuration, I simply type shedit <RETURN>.

When used without any arguments, the alias command lists any defined aliases (Apple doesn't provide any by default, so you probably won't see any if you try this and you haven't created any aliases of your own). If you use the alias command followed by a defined alias name, tcsh will show the expansion for that alias. If you decide you want to get rid of an alias, simply type unalias aliasname <RETURN> to remove it. These instructions for using aliases may sound complicated, but alias is actually quite simple and extremely useful. See Listing 15.4 for an example.

Listing 15.4: Using alias and unalias (the first line produces no response because no aliases exist yet)
Start example
[g4:~] power% alias
[g4:~] power% alias rm rm -i
[g4:~] power% alias l ls-F
[g4:~] power% alias sw ssh www
[g4:~] power% alias
l       ls-F
rm      (rm -i)
sw      (ssh www)
[g4:~] power% alias rm
rm -i
[g4:~] power% l
Desktop/   Library/   Music/      Public/
Documents/ Movies/    Pictures/   Sites/
[g4:~] power% unalias l
[g4:~] power% alias
rm      (rm -i)
sw      (ssh www)
End example
Note 

Note that if you use the alias command during a shell session, the alias you created is only valid until you log out. To make sure an alias is always available, you need to add it to your shell configuration file (see the sidebar "Making Permanent Changes to tcsh").

Wildcards

Wildcard expansion is an extremely useful shell feature (handled similarly in both sh-based and csh-based shells, fortunately). Wildcards are characters with special meaning to the shell; the two most common wildcards are the question mark, ?, which stands for exactly one instance of any character (like a Joker, which can substitute for any other card), and the asterisk, *, which can represent anything, from nothing to a sequence of different characters. When you type a command containing wildcard characters and press the return key to execute it, tcsh examines each word containing wildcards and computes all the valid filenames that match the typed pattern; it then replaces the wildcard word with all the matches alphabetically. That sounds far more complicated than wildcards really are; here's an example that illustrates the concept more clearly:

   [g4:~] power% ls /bin/*sh
   /bin/bash /bin/csh /bin/sh    /bin/tcsh /bin/zsh
   [g4:~] power% ls /bin/?sh
   /bin/csh /bin/zsh
   [g4:~] power% ls /bin/??sh
   /bin/bash /bin/tcsh
   [g4:~] power% ls /bin/bash /bin/tcsh
   /bin/bash /bin/tcsh

In this example, *sh means "any word of any length that ends in sh"; ?sh means "any three-letter word that ends in sh"; and ??sh means "any four-letter word that ends in sh."

Shell Scripts

If you type one or more valid commands into a text file and then save the file as myscript (just an example name), you can execute all of the commands contained in that file, one after another, by typing sh /pathtofile/myscript <RETURN> or tcsh /pathtofile/myscript <RETURN> (depending on what scripting syntax you used when writing the commands).

Tip 

To make a proper shell script that will garner the respect of Unix users everywhere, you should add comments describing what the script does and an initial line identifying which interpreter (shell) the script is written for, and give the script execute permission (using the command chmod u+x myscript <RETURN> in Terminal), so that it can be run as a normal program. To make sure your comments aren't interpreted by the shell as commands, begin each comment line with #. When the shell reads the script, it ignores everything after #, until the end of the line. In addition, the first line of a script is special; it should use the format #!/path to interpreter (i.e., #!/bin/sh for an sh script).

In fact, a significant proportion of the programs on a Unix system are just sophisticated scripts, as we can see by examining the programs in /usr/bin using the ls and file commands (Listing 15.5). In this example, we see that there are 552 programs (wc -l counts the number of lines in the listing), which includes over 50 sh scripts, over 400 compiled programs, and 40 perl scripts (grep -c counts the number of files that contain the designated text—which indicates each type of script—in the directory).

Listing 15.5: Types of Files in /usr/bin
Start example
[g4:~] power% ls /usr/bin | wc -l
552
[g4:~] power% file /usr/bin/* | grep -c Bourne
55
[g4:~] power% file /usr/bin/* | grep -c "C shell"
3
[g4:~] power% file /usr/bin/* | grep -c Mach-O
407
[g4:~] power% file /usr/bin/* | grep -c perl
40
End example

In fact, if you think way ba