Learning the Bash Shell

Learning Shell Command Syntax

Because a shell interprets what you type, it is important to know how the shell processes the text you enter. All shell commands have the following general format:

command option1 option2 ... optionN

A single line of command commonly is referred to as a command line. On a command line, you enter a command followed by one or more options (or arguments), known as command-line options (or command-line arguments). Sometimes the command-line options are also called switches.

Here are the basic command-line rules:

• Use a space or a tab to separate the command from the options.

• Separate individual options with a space or a tab.

• If you use an option that contains embedded spaces, you have to put that option inside quotation marks.

To search for my name in the password file, for example, I enter the grep command as follows:

grep "Naba Barkakati" /etc/passwd

When grep prints the line with my name, it looks like this:

naba:x:500:500:Naba Barkakati:/home/naba:/bin/bash

I use shadow passwords on my system (Chapter 22 describes shadow passwords). That's why the password field (the characters between the first and second colon) is a single letter x. Nevertheless, this line contains some useful information; the most interesting information (for the purposes of this discussion) is the field that follows the last colon (:). That field shows the name of the shell I am running.

The number and the format of the command-line options, of course, depend on the actual command. When you learn more about the commands, you see that the command-line options that control the behavior of a command are of the form -X , in which X is a single character.

Because various GNU tools and utilities implement most Linux commands, you should know that GNU command-line options begin with two dashes followed by a descriptive word. Thus, the GNU options are of the form -xxxx, where xxxx is a word denoting the option. For example, the GNU ls -all command shows all directory entries, including those that begin with a period (.). This is the same as the ls -a command in all UNIX systems.

If a command is too long to fit on a single line, you can press the backslash (\) key, followed by Enter. Then continue entering the command on the next line. In addition, you can concatenate several shorter commands on a single line by using the semicolon (;) as a separator between the commands. For example, when rebuilding the Linux kernel (as explained in Chapter 21), you can complete three sequential tasks by typing the following commands on a single line:

make dep; make clean; make zImage

Combining Commands

Linux follows the UNIX philosophy of giving the user a toolbox of many simple commands. You can, however, combine these simple commands to create a more sophisticated command. Suppose that you want to determine whether a device file named sbpcd resides in your system's /dev directory because you need that device file for a Sound Blaster Pro CD-ROM drive. You can use the command ls /dev to get a directory listing of the /dev directory and to see whether anything that contains sbpcd appears in the listing. Unfortunately, the /dev directory has a great many entries, and it may be difficult to locate any item that has sbpcd in its name. You can, however, combine the ls command with grep to come up with a command that does exactly what you want:

ls /dev | grep sbpcd

The shell sends the output of the ls command (the directory listing) to the grep command, which searches for the string sbpcd. That vertical bar (|) is known as a pipe because it acts as a conduit between the two programs; the output of the first command becomes the input of the second one.

For example, to see roughly how many processes running on your system, type:

ps ax | wc -l

This sends the output of the ps ax command (which lists all the processes) to the wc command that counts the number of lines and that count is one more than the total number of processes.

Here is another example of combining commands with pipes. Suppose you are having a meeting with a rather larger number of attendees and you have several sign-up sheets for attendees to put down their names. The attendees sign up on one or more of the sign-up sheets. Then, your assistant types each sheet into separate text files (named list1, list2, list3, and so on), one name per line. You can then combine the lists, sort the names, get rid of duplicates, and view the list with the following command:

cat list* | sort -u | more

The cat command simply turns all the names from the files into a stream of text. The sort -u command sorts the lines alphabetically and also removes any duplicates. Then more shows you the list a page at a time.

Using I/O Redirection

Linux commands designed to work together have a common feature-they always read from the standard input (usually, the keyboard) and write to the standard output (usually, the screen). Error messages are sent to the standard error (usually, the screen). These three devices often are referred to as stdout, stdin, and stderr.

If you want a command to read from a file, you can redirect the standard input to come from that file. Similarly, to save the output of a command in a file, redirect the standard output to a file. These features of the shell are called input and output redirection, or I/O redirection.

Using the following command, for example, you can search through all files in the /usr/include directory for the occurrence of the string typedef and save that list in a file called typedef.out:

grep typedef /usr/include/* > typedef.out

This command also illustrates another feature of Bash. When you use an asterisk (*), Bash replaces the asterisk with a list of all the filenames in the specified directory. Thus, /usr/include/* means all the files in the /usr/include directory.

Table 7-6 shows the syntax of common I/O redirection commands.

Using the I/O redirection commands is fairly straightforward. For example, suppose that you want to create a new file where you want to keep storing all text until you type ZZ. Here is how you can accomplish that task:

cat <<ZZ > input.txt

After you type this command, you can keep typing lines and type ZZ on a line when done. Everything you type should be saved in the file input.txt.

To save the list of files in the current directory in a file named filelist, type:

ls > filelist

To append the listing of the /etc/X11 ditrecory to the same filelist file, type:

ls /etc/X11 >> filelist

To alphabetically sort that list of files, you can feed it to the sort command, like this:

sort < filelist

You should see the sorted list of files scroll by on the screen. To save that sorted list in another file named sortedlist, type:

sort < filelist > sortedlist

Now, the sorted list of files are saved in the sortedlist files.

Understanding Environment Variables

The shell and other Linux commands need information to work properly. If you type a command that isn't one of that shell's built-in commands, the shell has to locate an executable file (whose name matches the command you type). The shell needs to know which directories to search for those files. Similarly, a text editor, such as vi, needs to know the type of terminal (even if the terminal happens to be a terminal window in GNOME or KDE).

One way to provide this kind of information to a program is through command-line options. However, if you use that approach, you may have to enter many options every time you start a program. UNIX provides an elegant solution through environment variables.

When you log in as a user, you get a set of environment variables that control many aspects of what you see and do on your Red Hat Linux system. If you want to see your current environment, go ahead and type the following command in a terminal window:

env

By the way, the printenv command also displays the environment, but env is shorter.

The env command should print a long list of lines. That whole collection of lines is the current environment and each line defines an environment variable. For example, here is a typical line displayed by the env command:

HOSTNAME=localhost.localdomain

This line defines the environment variable HOSTNAME, and it's defined as localhost. localdomain.

Secret

An environment variable is nothing more than a name associated with a string. On my system, for example, when I log in with the user name naba, the environment variable named PATH is defined as follows:

PATH=/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/naba/bin

The string to the right of the equal sign is the value of the PATH environment variable. By convention, the PATH environment variable is a sequence of directory names, each name separated from the preceding one by a colon (:).

When the shell has to search for a file, it simply searches the directories listed in the PATH environment variable. The shell searches the directories in PATH in the order of their appearance. Therefore, if two programs have the same name, the shell executes the one it finds first.

In a fashion similar to the shell's use of the PATH environment variable, an editor such as vi uses the value of the TERM environment variable to figure out how to display the file you are editing with vi. To see the current setting of TERM, type the following command at the shell prompt:

echo $TERM

If you type this command in a terminal window, the output is as follows:

xterm

To define an environment variable in Bash, use the following syntax:

export NAME=Value

Here, NAME denotes the name of the environment variable, and Value is the string representing its value. Therefore, you set TERM to the value vt100 by using the following command:

export TERM=vt100

You can set TERM to any of the terminal types defined in the /etc/termcap file, assuming, of course, that the terminal window or physical terminal matches the capabilities of that terminal type.

After you define an environment variable, you can change its value by simply specifying the new value with the syntax NAME=new-value. For example, to change the definition of TERM to ansi, type TERM=ansi at the shell prompt.

With an environment variable such as PATH, you typically want to append a new directory name to the existing definition, rather than define the PATH from scratch. The following example shows how to accomplish this task:

export PATH=$PATH:/sbin

This command appends the string :/sbin to the current definition of the PATH environment variable. The net effect is to add /sbin to the list of directories in PATH.

After you type that command, you can access programs in the /sbin directory such as ifconfig, a program that displays information about the network interfaces.

Note that you also can write this export command as follows:

export PATH=${PATH}:/sbin

PATH and TERM are only two of a handful of common environment variables. Table 7-7 lists some of the useful environment variables in Bash.

Viewing Process Information

Every time the shell acts on a command that you type, it starts a process. The shell itself is a process, so are any scripts or programs that the shell executes. Examples of such programs are the metacity window manager and Nautilus graphical shell in GNOME. You can use the ps command to see a list of processes. When you type ps ax, for example, Bash shows you the current set of processes. Following is a typical report displayed when you enter the ps ax command in a terminal window (I also include the -cols 256 option to ensure that you can see each command in its entirety):

ps ax --cols 256
  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:03 init
    2 ?        SW     0:00 [keventd]
    3 ?        SW     0:00 [kapmd]
    4 ?        SWN    0:00 [ksoftirqd_CPU0]
    7 ?        SW     0:00 [bdflush]
    5 ?        SW     0:01 [kswapd]
    6 ?        SW     0:00 [kscand]
    8 ?        SW     0:00 [kupdated]
    9 ?        SW     0:00 [mdrecoveryd]
   13 ?        SW     0:00 [kjournald]
   71 ?        SW     0:00 [khubd]
 1178 ?        SW     0:00 [kjournald]
... lines deleted ...
2601 ?        S      0:00 in.telnetd: dhcppc3
2602 ?        S      0:00 login -- naba
2603 pts/1    S      0:00 -bash
2715 pts/1    R      0:00 ps ax --cols 256

In the default output format, the COMMAND column shows the commands that create the processes. This list shows the bash shell and the ps command as the processes. Other processes include all the programs the shell starts when I log in at the graphical login screen and start a GNOME session. In particular, the list includes the metacity (window manager) and the nautilus (graphical shell) processes.

Secret

The ps ax form of the command is the BSD UNIX format of the ps command. You also can write the command as ps -ax-with a hyphen prefix for the options as in most other Linux commands. The Linux ps command accepts options both with and without a hyphen prefix. In a typical output of the ps ax command you should expect to see anywhere from 60 to 70 processes in the list. Quite a few processes are running in your Red Hat Linux system even when you are the only user on the system. Hare are the first two lines of output from ps ax:

  PID TTY      STAT   TIME COMMAND
    1 ?        S      0:04 init

In the output of the ps command, the first column has the heading PID and it shows a number for each process. PID stands for process ID (identification), which is a sequential number the Linux kernel assigns. If you look through the output of the ps ax command, you should see that the init command is the first process; it has a PID or process number of 1. That's why init is referred to as the mother of all processes.

The process ID or process number is useful when you have to stop an errant process forcibly. Look at the output of the ps ax command, and note the PID of the offending process. Then use the kill command with that process number. To stop process number 12318, for example, type kill -9 12318.

UNIX systems, including Linux, use signals to notify a process that a specific event has occurred. The kill command enables you to send a signal to a process (identified by a process number). The -9 part of the kill command indicates the signal to be sent; 9 happens to be the number of the SIGKILL signal that causes a process to exit.

Running Commands in the Background or in Virtual Consoles

When using MS-DOS, you have no choice but to wait for each command to complete before you enter the next command. (You can type ahead a bit, but the MS-DOS system can hold only a few characters in its internal buffer.) Linux, however, can handle multiple tasks simultaneously. The only problem you may have is that the terminal or console is tied up until a command completes.

If you work in a terminal window and a command takes too long to complete, you can open another terminal window and continue to enter other commands. If you work in text mode, however, and a command seems to take too long, you need some other way to access your system.

Several methods enable you to continue working while your Linux system handles a lengthy task:

• You can start a lengthy command in the background, which means that the shell starts the process corresponding to a command and immediately returns to accept more commands. The shell does not wait for the command to complete; the command runs as a distinct process in the background. To start a process in the background, simply place an ampersand (&) at the end of a command line. When I want to run the convpcx shell script to convert an image file named image1 to PCX format, for example, I run the script in the background by using the following command:

  convpcx image1 &

• If you want a command to continue running in the background after you log out, use nohup to start the command using the following syntax:

  nohup command
  

  This causes the command to run in the background and all output is redirected to a file named nohup.out.

• If a command (that you have not run in the background) seems to be taking a long time, press Ctrl-Z to stop it; then type bg to put that process in the background.

• Use the virtual-console feature of Linux. Even though your Linux system has only one physical terminal or console (the combination of monitor and keyboard is called the terminal or console), it gives you the appearance of having multiple consoles. The initial text screen is the first virtual console. Press Alt-F2 to get to the second virtual console, Alt-F3 for the third virtual console, and so on. From the GUI desktop, you have to press Ctrl-Alt-F1 to get to the first virtual console, Ctrl-Alt-F2 for the second one, and so on.

  Insider Insight

  To get back to the GUI desktop, press Ctrl-Alt-F7. You can use one of the virtual consoles to log in and kill processes that may be causing your X display screen to become unresponsive (for instance, if the mouse stops responding).

Typing Less with Filename Completion

Many commands take a filename as an argument. When you want to browse through a file named /etc/X11/XF86Config, for example, type the following:

more /etc/X11/XF86Config

That entry causes the more command to display the file /etc/X11/XF86Config one screen at a time. For commands that take a filename as an argument, Bash includes a feature that enables you to type short filenames. All you have to type is the bare minimum-just the first few characters-to identify the file uniquely in its directory.

To see an example, type more /etc/X11/XF, but don't press Enter; press Tab instead. Bash automatically completes the filename, so that the command becomes more /etc/X11/XF86Config. Now, press Enter to run the command.

Using Wildcards in Filenames

Another way to avoid typing too many filenames is to use wildcards, special characters, such as the asterisk (*) and question mark (?), that match zero or more characters in a string. If you are familiar with MS-DOS, you may use commands such as COPY *.* A: to copy all files from the current directory to the A drive. Bash accepts similar wildcards in filenames. In fact, Bash provides many more wildcard options than MS-DOS does.

Bash supports three types of wildcards:

• The asterisk (*) character matches zero or more characters in a filename. Therefore, * denotes all files in a directory.

• The question mark (?) matches any single character.

• A set of characters in brackets matches any single character from that set. The string [xX]*, for example, matches any filename that starts with x or X.

Wildcards are handy when you want to perform a task on a group of files. To copy all the files from a directory named /mnt/cdrom to the current directory, for example, type the following:

cp /mnt/cdrom/* .

Bash replaces the wildcard character * with the names of all the files in the /mnt/cdrom directory. The period at the end of the command represents the current directory.

You can use the asterisk with other parts of a filename to select a more specific group of files. Suppose that you want to use the grep command to search for the string typedef struct in all files of the /usr/include directory that meet the following criteria:

• The filename starts with s.

• The filename ends with .h.

The wildcard specification s*.h denotes all filenames that meet these criteria. Thus, you can perform the search by using the following command:

grep "typedef struct" /usr/include/s*.h

The string contains a space that you want the grep command to find, so you have to enclose that string in quotation marks. This method ensures that Bash does not try to interpret each word in the string as a separate command-line argument.

Although the asterisk (*) matches any number of characters, the question mark (?) matches a single character. Suppose that you have four files-image1.pcx, image2.pcx, image3.pcx, and image4.pcx-in the current directory. To copy these files to the /mnt/floppy directory, use the following command:

cp image?.pcx /mnt/floppy

Bash replaces the single question mark with any single character and copies the four files to /mnt/floppy.

The third wildcard format-[...]-matches a single character from a specific set. You may want to combine this format with other wildcards to narrow the matching filenames to a smaller set. To see a list of all filenames in the /etc/X11/xdm directory that start with x or X, type the following command:

ls /etc/X11/xdm/[xX]*

Viewing the Command History

To make it easy for you to repeat long commands, Bash stores up to 500 old commands. Essentially, Bash maintains a command history (a list of old commands). To see the command history, type history. Bash displays a numbered list of the old commands, including those you have entered during previous logins. That list may resemble the following:

     1  cd
     2  ls -a
     3  more /etc/X11/XF86Config
     4  history

If the command list is very long, you may choose to see only the last few commands. To see the last 10 commands only, type the following:

history 10

To repeat a command from the list the history command has generated, simply type an exclamation point (!), followed by that command's number. To repeat command 3, type !3.

You also can repeat an old command without knowing its command number. Suppose that you typed more /usr/lib/X11/xdm/xdm-config a while ago, and now you want to look at that file again. To repeat the previous more command, type the following:

!more

Often, you may want to repeat the last command you typed, perhaps with a slight change. For example, you may have displayed the contents of the directory by using the ls -l command. To repeat that command, type two exclamation points as follows:

!!

Sometimes, you may want to repeat the previous command but add extra arguments to it. Suppose that ls -l shows too many files. Simply repeat that command, but pipe the output through the more command as follows:

!! | more

Bash replaces the two exclamation points with the previous command and appends | more to that command.

Editing Recalled Commands

After you recall a command, you do not have to use the command as is; you can edit the command. Bash supports a wide variety of command-line editing commands. These commands resemble those the emacs and vi editors use.

Suppose that you want to look at the file /etc/X11/XF86Config, but you type the following:

more /etc/X11/XF86config

After you press Enter and see an error message stating that no such file exists, you realize that the c in config should have been uppercase. Instead of retyping the entire line, you can type the following editing command to fix the problem:

^con^Con

Bash interprets this command to mean that it should replace the string con with Con in the previous command.

Secret

By default, Bash enables you to edit the command line by using a small subset of commands that the emacs editor supports.

Because I am already familiar with emacs, I use emacs commands to edit commands. To bring back a previous command line, for example, I press Ctrl-P. Then I use the following keystrokes to edit the command line:

• Ctrl-B to go backward a character

• Ctrl-F to go forward a character

• Ctrl-D to delete the character on which the text cursor rests

  I can also recall previous command lines by pressing the up arrow and then edit the command line. To move about the command line, I can use the left and right arrow keys.

  To insert text, I type the text. Although emacs has a huge selection of editing commands, you can edit Bash command lines adequately with the preceding small set.

  By default, the Bash shell supports emacs-style editing of command lines. To edit the command line by using the vi editor's commands, type set -o vi at the shell prompt. In vi mode, you have to press Esc to enter editing commands and press i to switch back to interacting with the shell. From vi mode, you can return to emacs editing by typing set -o emacs at the shell prompt.

Using Aliases

While configuring the Linux kernel and creating a new Linux boot floppy, I found myself changing the directory to /usr/src/linux-2.4/arch/i386/boot quite a few times. After typing that directory name twice, I immediately set up a shortcut using Bash's alias feature:

alias goboot='cd /usr/src/linux-2.4/arch/i386/boot'

I intentionally did not use any underscore characters or uppercase letters in goboot because I wanted the alias to be quick and easy to type (and it had to mean something to me only). Now that I've defined the alias, I can go to that directory by typing the following at the Bash prompt:

goboot

As you can see, an alias simply is an alternate (and usually shorter) name for a lengthy command. Bash replaces the alias with its definition and performs the equivalent command.

Secret

Many users use the alias feature to give more familiar names to common commands. If you are familiar with DOS/Windows commands and you use the dir command to get a directory listing, you can simply define dir as an alias for ls (the Linux command that displays the directory listing), as follows:

alias dir=ls

Now, you can type dir whenever you want to see a directory listing.

Another good use of an alias is to redefine a dangerous command, such as rm, to make it safer. By default, the rm command deletes one or more specified files. If you type rm * by mistake, rm deletes all files in your current directory. I learned this the hard way one day when I wanted to delete all files that ended with .xwd. (These files contained old screen images I no longer needed.) I intended to type rm *.xwd, but somehow I ended up typing rm * .xwd. I got the following message:

rm: .xwd: No such file or directory

At first I was puzzled by the message, so I typed ls to see the directory's contents again. When the listing showed nothing, I realized that I had an extra space between the * and .xwd. Of course, all the files in that directory were gone forever.

The rm command provides the -i option, which asks for confirmation before deleting a file. To make that option a default, add the following alias definition to the .bash_profile file in your home directory:

alias rm='rm -i'

That's it! From now on, when you use rm to delete a file, the command first asks for confirmation, as follows:

rm .bash_profile
rm: remove `.bash_profile'? n

Press y to delete the file; otherwise, press n.

If you type the same long command often, you should define an alias for that command. To make sure that the alias is available whenever you log in, place the definition in the .bash_profile file in your home directory.