Section 5.2. Startup Files

The kernel configuration brings the basic transport and IP datagram services of TCP/IP into Unix. But there is much more to the TCP/IP suite than just the basic services. How are these other protocols included in the Unix configuration?

Some protocols are explicitly started by including them in the boot files. This technique is used, for example, to start the Routing Information Protocol (RIP) and the Domain Name System (DNS). Network services that either have a long startup procedure or are in constant demand are normally started by a script at boot time, and run as daemon processes the entire time the system is running.

Anything that can be run from a shell prompt can be stored in a file and run as a shell script. Systems use this capability to automate the startup of system services. There are two basic Unix startup models that control how startup files are invoked: the BSD model and the System V model.

The BSD model is the simplest: a limited number of startup scripts are executed in order every time the system boots. At its simplest, there are three basic scripts, /etc/rc, /etc/rc.boot, and /etc/rc.local, executed in that order for system initialization, service initialization, and local customization. On BSD Unix systems, network services are usually started by the /etc/rc.boot file or the /etc/rc.local file.

On systems that use the BSD startup model, place customized network configuration commands in the rc.local script. rc.local executes at the end of the startup process. Any configuration values set in this file override the earlier configuration commands.

The BSD startup model is used on BSD systems and SunOS systems. Linux and Solaris systems use the System V startup model. The System V startup model employs a much more complex set of startup files.[8] This model uses whole directories of scripts executed by the init process, with different directories being used depending on the runlevel of the system.

[8] A good description of the maze of System V initialization files is provided in Essential System Administration by Æleen Frisch (O'Reilly & Associates).

5.2.1 Startup Runlevels

To understand System V startup, you need to understand runlevels, which are used to indicate the state of the system when the init process is complete. There is nothing inherent in the system hardware that recognizes runlevels; they are purely a software construct. init and /etc/inittab -- the file used to configure initare the only reasons why the runlevels affect the state of the system. We use Red Hat Linux as an example of how runlevels are used.

Linux defines several runlevels that run the full gamut of possible system states from not-running (halted) to running multiple processes for multiple users:

  • Runlevel 0 shuts down all running processes and halts the system.

  • Runlevel 1 places the system in single-user mode. Single-user mode is used by the system administrator to perform maintenance that cannot be done when users are logged in. This runlevel may also be indicated by the letter S instead of the number 1. Solaris uses S for single-user mode.

  • Runlevel 2 is a special multiuser mode that does not support file sharing.

  • Runlevel 3 provides full multiuser support with the full range of services, including NFS file sharing. It is the default mode used on Solaris systems.

  • Runlevel 4 is unused. You can design your own system state and implement it through runlevel 4.

  • Runlevel 5 initializes the system as a dedicated X Windows terminal. Linux systems use this to provide an X Windows console login. When Linux systems boot at runlevel 3, they provide a text-based console login. Solaris does not use this runlevel. Entering runlevel 5 on a Solaris system causes a system shutdown.

  • Runlevel 6 shuts down all running processes and reboots the system.

As these notes make clear, different systems use the same runlevels in different ways. That is because runlevels are software. They are boot command arguments that tell init which startup scripts should be run. The scripts that are run can contain any valid commands. init maps runlevels to startup scripts using the inittab file. Understanding /etc/inittab

All of the lines in the inittab file that begin with a sharp sign (#) are comments. A liberal dose of comments is needed because the syntax of inittab configuration lines is terse and arcane. An inittab entry has this general format:


The label is a one- to four-character tag that identifies the entry. Because some systems support only two-character labels, most configurations limit all labels to two characters. The labels can be any arbitrary character string; they have no intrinsic meaning.

The runlevel field indicates the runlevels to which the entry applies. For example, if the field contains a 3, the process identified by the entry must be run for the system to initialize runlevel 3. More than one runlevel can be specified. Entries that have an empty runlevel field are not involved in initializing specific runlevels. For example, Linux systems have an inittab entry to handle the three-finger salute (Ctrl+Alt+Del); it does not have a value in the runlevel field.

The action field defines the conditions under which the process is run. Table 5-1 lists the action values used on Red Hat, Mandrake, and Caldera Linux systems.

Table 5-1. Linux inittab action values




Runs when the system boots. Runlevels are ignored.


Runs when the system boots, and init waits for the process to complete. Runlevels are ignored.


Runs when Ctrl+Alt+Del is pressed, which passes the SIGINT signal to init. Runlevels are ignored.


Doesn't execute a process. It sets the default runlevel.


Runs when init receives a signal from the keyboard. This requires that a key combination be mapped to KeyBoardSignal.


Disables the entry so the process is not run.


Runs one time for every runlevel.


Runs when the system enters one of the special runlevels A, B, or C.


Runs when init receives the SIGPWR signal.


Runs when init receives the SIGPWR signal and the file /etc/powerstatus contains the word OK.


Runs when init receives the SIGPWR signal, and init waits for the process to complete.


Restarts the process whenever it terminates.


Runs before any boot or bootwait processes.


Runs the process upon entering the run mode, and init waits for the process to complete.

The last field in an inittab entry is process. It contains the process that init executes. The process appears in the exact format that it is executed from the command line. Therefore the process field starts with the name of the program that is to be executed followed by the arguments that will be passed to that process. For example, /sbin/shutdown -t3 -r now, which is the process executed by some Linux systems when Ctrl+Alt+Del is pressed, is the same command that could be typed at the shell prompt to reboot the system. On most inittab entries, the process field contains the name of a startup script. Two main types of startup scripts are used: the system initialization script and the runlevel initialization scripts. These sample lines from a Red Hat Linux system show both:

# System initialization.


l0:0:wait:/etc/rc.d/rc 0

l1:1:wait:/etc/rc.d/rc 1

l2:2:wait:/etc/rc.d/rc 2

l3:3:wait:/etc/rc.d/rc 3

l4:4:wait:/etc/rc.d/rc 4

l5:5:wait:/etc/rc.d/rc 5

l6:6:wait:/etc/rc.d/rc 6

These seven lines are the real heart of the inittab filethey invoke the startup scripts. The first line tells init to run the boot script located at /etc/rc.d/rc.sysinit to initialize the system. This entry has no runlevel value. It is run every time the system starts. The system initialization script performs certain essential tasks. For example, the Red Hat rc.sysinit script:

  • Initializes the swap space

  • Runs the filesystem check

  • Mounts the /proc filesystem

  • Mounts the root filesystem as read-write after the fsck completes

  • Loads the loadable kernel modules

Other initialization scripts may look different than Red Hat's, but they perform very similar functions. For example, a Caldera system begins by loading the loadable modules. It then activates the swap space, does the filesystem check, and remounts the root filesystem as read-write. The order is different, but the major functions are the same.

After the system initialization script is run, init runs a script for the specific runlevel. The remaining six lines in the sample are used to invoke the startup scripts for individual runlevels. Except for the runlevel involved, each line is identical.

Let's use the line with label l3 as an example. This line starts all of the processes and services needed to provide the full multiuser support. The runlevel is 3. The action wait directs init to wait until the startup script terminates before going on to any other entries in the inittab file that relate to runlevel 3. init executes the script /etc/rc.d/rc and passes that script the command-line argument 3.

The control script, /etc/rc.d/rc, then runs all the scripts that are appropriate for the runlevel. It does this by running the scripts that are stored in the directory /etc/rcn.d, where n is the specified runlevel. In our example, the rc script is passed a 3, so it runs the scripts found in the directory /etc/rc.d/rc3.d. A listing of that directory from a Red Hat system shows that there are lots of scripts:

$ ls /etc/rc.d

init.d  rc0.d  rc2.d  rc4.d  rc6.d     rc.sysinit

rc      rc1.d  rc3.d  rc5.d  rc.local

$ ls /etc/rc.d/rc3.d

K03rhnsd      K35smb       K74ntpd    S05kudzu     S25netfs       S85httpd

K16rarpd      K45arpwatch  K74ypserv  S06reconfig  S26apmd        S90crond

K20nfs        K45named     K74ypxfrd  S08ipchains  S28autofs      S90xfs

K20rstatd     K50snmpd     K75gated   S09isdn      S40atd         S95anacron

K20rusersd    K50tux       K84bgpd    S10network   S55sshd        S99linuxconf

K20rwalld     K55routed    K84ospf6d  S12syslog    S56rawdevices  S99local

K20rwhod      K61ldap      K84ospfd   S13portmap   S56xinetd

K28amd        K65identd    K84ripd    S14nfslock   S60lpd

K34yppasswdd  K73ypbind    K84ripngd  S17keytable  S80sendmail

K35dhcpd      K74nscd      K85zebra   S20random    S85gpm

The scripts that begin with a K are used to kill processes when exiting a specific runlevel. In the listing above, the K scripts would be used when terminating runlevel 3. The scripts that start with an S are used when starting runlevel 3. None of the items in rc3.d, however, is really a startup script. They are logical links to the real scripts, which are located in the /etc/rc.d/init.d directory. For example, S80sendmail is linked to init.d/sendmail. This raises the question of why the scripts are executed from the directory rc3.d instead of directly from init.d where they actually reside. The reasons are simple. The same scripts are needed for several different runlevels. Using logical links, the scripts can be stored in one place and still be accessed by every runlevel from the directory used by that runlevel.

Scripts are executed in alphabetical order. Thus S10network is executed before S80sendmail. This allows the system to control the order in which scripts are executed through simple naming conventions. Different runlevels can execute the scripts in different orders while still allowing the real scripts in init.d to have simple, descriptive names. A listing of the init.d directory shows these descriptive names:

$ ls /etc/rc.d/init.d

amd       functions  kdcrotate  network  rarpd       rwalld    xfs

anacron   gated      keytable   nfs      rawdevices  rwhod     xinetd

apmd      gpm        killall    nfslock  reconfig    sendmail  ypbind

arpwatch  halt       kudzu      nscd     rhnsd       single    yppasswdd

atd       httpd      ldap       ntpd     ripd        smb       ypserv

autofs    identd     linuxconf  ospf6d   ripngd      snmpd     ypxfrd

bgpd      ipchains   lpd        ospfd    routed      sshd      zebra

crond     iptables   named      portmap  rstatd      syslog

dhcpd     isdn       netfs      random   rusersd     tux

It is possible to place a customized configuration command directly in the applicable script in the init.d directory. A better alternative on a Red Hat system is to place any local changes in rc.local.

Like BSD systems, Linux systems provide an rc.local script for local customization. In general, you do not directly edit boot scripts. The exception to this rule is the rc.local script located in the /etc/rc.d directory. It is the one customizable startup file, and it is reserved for your use; you can put anything you want in there. After the system initialization script executes, the runlevel scripts execute in alphabetical order. The last of these is S99local, which is a link to rc.local. Since it is executed last, the values set in the rc.local script override other configuration values.

Solaris also uses the System V startup model, but it makes things a little more difficult than Linux does. First off, it does not provide an rc.local script. If you want to use one, you need to add your own to the runlevel directories. Secondly, Solaris does not use many logical links in the runlevel directories. Therefore, there is no guarantee of a central place to modify scripts that are used for all runlevels. Additionally, each runlevel has a separate controlling script that can introduce differences in the startup process for each runlevel. For example, /sbin/rc2 is the controlling script for runlevel 2 and /sbin/rc3 is the controlling script for runlevel 3. All of these differences make the Solaris startup process more complex to analyze.

On a Solaris 8 system, runlevel 3 is the default runlevel for a multiuser system offering network services. The /sbin/rc3 controlling script runs the scripts in /etc/rc2.d and then those in /etc/rc3.d. Basic network configuration is handled in /etc/rc2.d by the S69inet script and the S72inetsvc script. Several other scripts in both /etc/rc2.d and /etc/rc3.d are involved in launching network services.

For troubleshooting purposes it is important to understand where and how things happen during the system startup. When the network fails to initialize properly, it is good to know where to look. However, when you configure the network you should stick with the standard tools and procedures provided with your system. Directly modifying startup scripts can cause problems during the startup and can lead to lots of confusion for the other people who help you maintain your systems.

Of course, not all network services are started by a boot script. Most network services are started on demand. The most widely used tool for starting network services on demand is inetd, the Internet Daemon.