17.2 Updating System Software

Several factors go into the selection of a particular Unix operating system or application and the choice of which version of the software to run, but we won't try to get into those issues here.[6] Once you've settled on which version of which software you're going to install, however, it is imperative that you ensure that patches are available for all known security problems in the software, that you find those patches, and that you apply them. Similarly, once the system is up and running, you must be vigilant in learning about newly discovered security problems in your operating system and applications so you'll be able to apply patches for them as they become available.

[6] Except to urge you to consider the security features of the software as part of your decision-making process, of course.

Installing a New Unix System

Because of the speed with which attackers discover new systems on the Internet, and the easy availability of attack scripts that can determine the operating system version of a remote Unix host and tailor its attacks against known bugs in that version, you must not put an unpatched Unix host onto the Internet in a way that makes it accessible to outsiders. Some commercial Unix vendors provide patch CD-ROMs quarterly on a subscription basis, but the very latest patches can usually be obtained only from the vendor's web site.

The most secure way to patch a new installation is to download the patches to another computer that's already connected to the Internet and updated with the latest security patches (perhaps a Mac or PC client that runs no server services). Once downloaded, they can be burned onto a CD-ROM or transferred to the new system using a local network connection, and then applied.

If no other Internet-connected host is available or suitable, the new host may have to be connected before the patches are applied. In this case, disable all network servers on the machine, and make the connection as brief as possible?only long enough to download the required patches?and then physically remove the machine from the network while the patches are applied. This process can be made even more secure if the machine's connection can be protected by a stateful firewall or a router that implements network address translation, so that the only packets that can reach the new host are those associated with a connection initiated by the new host.

You can't stay up to date with software that you don't know you've installed. An important component of any ongoing updating process is to inventory your system and keep track of new applications that you've installed. Operating systems that use packages usually provide commands that will let you determine which packages you have installed. Source-based software management typically relies on keeping all of the source code to the installed applications in a single location where it can be easily found.

17.2.1 Learning About Patches

There are several avenues for learning about security problems and patches for operating systems and applications:

  • Every Unix operating system and most major applications, such as web servers, have an associated mailing list for announcements of new versions. Many maintain a separate list for announcements of security-related issues. Subscribe to these lists and pay attention to the messages

  • Several mailing lists collect and distribute security alerts for many products. These are listed in Appendix D. Subscribe to these lists (perhaps in digest form) and pay attention to the messages.

  • If your vendor provides a subscription patch CD-ROM service, consider subscribing. Although these CD-ROMs may not provide up-to-the-minute patches, they can save a lot of time when you are bringing up a new system by reducing the number of patches that need to be downloaded.

  • Automatic update systems compare installed packages with the latest versions of packages available on the vendor's web site and report which packages are out of date. Also, most can be configured to automatically download and install the upgraded packages. This can be useful if you don't change your configuration from the vendor defaults, and you trust the vendor to upgrade your system. Some can be run periodically through cron; others must be run manually on a regular basis. Here's an example of searching for security updates in Debian GNU/Linux:[7]

    [7] Linux distributions are particularly fond of this mechanism. Red Hat provides a system called up2date, SuSE provides Yast Online Update, Debian provides apt-get, and RPM-based distributions can take advantage of third-party software such as AutoRPM to automate updates.

    # apt-get update                   (Update the current list of available packages)
    Hit http://security.debian.org stable/updates/main Packages
    Hit http://security.debian.org stable/updates/main Release
    Hit http://security.debian.org stable/updates/contrib Packages
    Hit http://security.debian.org stable/updates/contrib Release
    Hit http://security.debian.org stable/updates/non-free Packages
    Hit http://security.debian.org stable/updates/non-free Release
    Get:1 http://security.debian.org woody/updates/main Packages [58.8kB]
    Get:2 http://security.debian.org woody/updates/main Release [110B]
    Get:3 http://security.debian.org woody/updates/contrib Packages [1490B]
    Get:4 http://security.debian.org woody/updates/contrib Release [113B]
    Get:5 http://security.debian.org woody/updates/non-free Packages [29B]
    Get:6 http://security.debian.org woody/updates/non-free Release [114B]
    Fetched 60.7kB in 0s (65.2kB/s)
    Reading Package Lists... Done
    Building Dependency Tree... Done
    # apt-get -u upgrade            (Upgrade to latest packages with user confirmation)
    langit:/home/alansz# apt-get -u upgrade
    Reading Package Lists... Done
    Building Dependency Tree... Done
    The following packages will be upgraded
    1 packages upgraded, 0 newly installed, 0 to remove and 0  not upgraded.
    Need to get 166kB of archives. After unpacking 221kB will be used.
    Do you want to continue? [Y/n] y
    Get:1 http://security.debian.org woody/updates/main nis 3.9-6.1 [166kB]
    Fetched 166kB in 1s (158kB/s)
    Preconfiguring packages ...
    (Reading database ... 33243 files and directories currently installed.)
    Preparing to replace nis 3.9-6 (using .../archives/nis_3.9-6.1_i386.deb) ...
    Unpacking replacement nis ...
  • Finally, you can manually check the vendor's web site on a regular basis for new versions of software.

17.2.2 Upgrading Distributed Applications

Under package management systems, upgrading a package is usually a very simple procedure. For example, here's what you do to upgrade the bzip2-devel package on a system that uses the RPM package manager:

# ls -l *.rpm 
-rw-r--r--    1 root     root        33708 Apr 16 23:15 bzip2-devel-1.0.2-2.i386.rpm
# rpm -K bzip2-devel-1.0.2-2.i386.rpm       Check the package's integrity (checksum and
bzip2-devel-1.0.2-2.i386.rpm: md5 OK
# rpm -Uvh bzip2-devel-1.0.2-2.i386.rpm   Upgrade the package.
Preparing...      ########################################### [100%]
   1:bzip2-devel  ########################################### [100%]
# rpm -q bzip2-devel                Confirm that the installed version is now 1.0.2-2.

Installing a Solaris security patch is similarly easy. After we downloaded patch 104489-15.tar.Z from http://sunsolve.sun.com/ we used the installpatch script bundled inside the patch archive to install the appropriate patch:

Checksums and Signatures

RPM packages incorporate two features to help verify their integrity. Every RPM file includes, as metadata, the MD5 cryptographic checksum of the packaged software. This checksum can detect file corruption or tampering. In addition, RPM files can be cryptographically signed using the GNU Privacy Guard (gpg) public key encryption software.

The -K (or ?checksig) option to the rpm command performs integrity checking. It computes the MD5 checksum of the packaged software and compares it to the included checksum, and it tests the gpg signature for validity:

% rpm -K nfs-utils-0.3.3-5.i386.rpm
nfs-utils-0.3.3-5.i386.rpm: md5 OK

The word md5 implies that the MD5 check succeeded. The OK means that the gpg signature was verified. Adding the -v option provides additional useful output, notably information about exactly which key the package was signed with:

% rpm -K -v nfs-utils-0.3.3-5.i386.rpm
MD5 sum OK: 4e3d971e40e494764d0639600bef86ce
gpg: Warning: using insecure memory!
gpg: Signature made Tue Apr 16 23:44:58 2002 CDT using DSA key ID DB42A60E
gpg: Good signature from "Red Hat, Inc <security@redhat.com>"

This package was signed by Red Hat's security key (or something purporting to be that key?naturally, you must ensure that your gpg public keyring contains keys you trust for signing software).[8]

Debian's package management system also incorporates gnupg signatures.

[8] The "insecure memory" warning is produced because gpg is not installed setuid root on this system, and thus cannot lock memory pages to prevent them from being paged to disk. This is a concern when you're using secret keys that might be revealed, but is not a concern if you are using only public keys to verify software.

% ls *.tar.Z
% uncompress *.Z
% su
Password: password

# tar xf 104489-15.tar
# cd 104489-15
# ls
.diPatch*          SUNWtltk/          backoutpatch*      postbackout*
Install.info*      SUNWtltkd/         installpatch*      postpatch*
README.104489-15   SUNWtltkm/         patchinfo*
# ./installpatch .
Checking installed patches...
Generating list of files to be patched...
Verifying sufficient filesystem capacity (exhaustive method)...
Installing patch packages...

Patch number 104489-15 has been successfully installed.
See /var/sadm/patch/104489-15/log for details
Executing postpatch script...

Patch packages installed:

# showrev -p | egrep 104489
Patch: 104489-01  Obsoletes:   Packages: SUNWtltk, SUNWtltkd
Patch: 104489-14  Obsoletes:   Packages: SUNWtltk, SUNWtltkd, SUNWtltkm
Patch: 104489-15  Obsoletes:   Packages: SUNWtltk, SUNWtltkd, SUNWtltkm

If you're using source-based management, upgrading involves either performing a CVS checkout of the updated source code or applying a patch file to the old source code to update it. In either case, the source code must then be recompiled and reinstalled. Here is an example of applying a patch to an application:

% ls -ld * 
-rw-rw----    1 dunemush dunemush   188423 Jul 20 12:07 1.7.5-patch09
drwx------   10 dunemush dunemush     4096 Jul  4 16:15 pennmush/
% cd pennmush 
% patch -p1 -s < ../1.7.5-patch09 
% make 
  . . . source code compile messages . . . 
% make install 
  . . . installation messages . . . 

If you're upgrading a server program, of course, you will need to stop the running server process and restart it to run the newly installed version?simply changing the server program on disk is not sufficient!

Backing Out and Backing Up

Not every upgrade is a panacea. Sometimes upgrades cause more problems than they solve, either because they break important functionality, or because they don't provide the desired fix. It's important to be able to revert to the pre-upgrade software if the upgrade proves troublesome.

There are two basic strategies for recovering from a bad upgrade. First, it may be possible to "back out" the patch and reinstall the earlier version. Under source-based management systems, the patch program can also be used to remove a previously applied patch, or the earlier version can be checked out from a CVS repository. It can be more difficult to cleanly back out a package. Although most package management software provides a way to overwrite an installed package with an earlier version, if the package dependencies have also been updated, older versions of the dependencies may also have to be located and installed.

A second strategy for source-based systems is to locally back up older versions of software. By keeping older versions of source code, it's generally not difficult to reinstall the earlier version. Multiple versions can be kept in separate directories in /usr/src, or a version control system such as RCS or CVS can be used locally to track multiple versions of software in the same directory.

Perhaps the most reliable method is to perform a full backup of your system prior to the changes. Then, if the upgrade goes badly, you can restore your system to the prior state (see Chapter 18).

17.2.3 Sensitive Upgrades

Some upgrades are best performed when the system is in single-user mode. Although upgrading an application does not generally affect running processes, there are exceptions. Some programs dynamically load object code while running, and upgrading such programs without first stopping them can cause problems if the older version of the process loads the newer version of the dynamic code.

Similar caveats apply to updating shared libraries, and especially the shared C library. Upgrades to core shared libraries should always be installed in single-user mode.

Kernel upgrades also warrant single-user mode: when installing the new kernel, the new kernel will necessarily overwrite the old kernel. Installing a new kernel requires a system reboot anyway, however, so dropping down to single-user mode is not much more of an inconvenience.

    Part VI: Appendixes