Section 3.1. OS Hardening Principles

Operating-system hardening can be time consuming and even confusing. Like many OSes designed for a wide range of roles and user levels, Linux has historically tended to be "insecure by default": most distributions' default installations are designed to present the user with as many preconfigured and active applications as possible. Therefore, securing a Linux system not only requires you to understand the inner workings of your system; you may also have to undo work others have done in the interest of shielding you from those inner workings!

Having said that, the principles of Linux hardening in specific and OS hardening in general can be summed up by a single maxim: "that which is not explicitly permitted is forbidden." As I mentioned in the previous chapter, this phrase was coined by Marcus Ranum in the context of building firewall rules and access-control lists. However, it scales very well to most other information security endeavors, including system hardening.

Another concept originally forged in a somewhat different context is the Principle of Least Privilege. This was originally used by the National Institute of Standards and Technology (NIST) to describe the desired behavior of the "Role-Based Access Controls" it developed for mainframe systems: "a user [should] be given no more privilege than necessary to perform a job" (http://hissa.nist.gov/rbac/paper/node5.html).

Nowadays people often extend the Principle of Least Privilege to include applications; i.e., no application or process should have more privileges in the local operating environment than it needs to function. The Principle of Least Privilege and Ranum's maxim sound like common sense (they are, in my opinion). As they apply to system hardening, the real work stems from these corollaries:

  • Install only necessary software; delete or disable everything else.

  • Keep all system and application software painstakingly up-to-date, at least with security patches, but preferably with all package-by-package updates.

  • Delete or disable unnecessary user accounts.

  • Don't needlessly grant shell access: /bin/false should be the default shell for nobody, guest, and any other account used by services, rather than by an individual local user.

  • Allow each service (networked application) to be publicly accessible only by design, never by default.

  • Run each publicly accessible service in a chrooted filesystem (i.e., a subset of /).

  • Don't leave any executable file needlessly set to run with superuser privileges, i.e., with its SUID bit set (unless owned by a sufficiently nonprivileged user).

  • If your system has multiple administrators, delegate root's authority.

  • Configure logging and check logs regularly.

  • Configure every host as its own firewall; i.e., bastion hosts should have their own packet filters and access controls in addition to (but not instead of) the firewall's.

  • Check your work now and then with a security scanner, especially after patches and upgrades.

  • Understand and use the security features supported by your operating system and applications, especially when they add redundancy to your security fabric.

  • After hardening a bastion host, document its configuration so it may be used as a baseline for similar systems and so you can rebuild it quickly after a system compromise or failure.

All of these corollaries are ways of implementing and enforcing the Principle of Least Privilege on a bastion host. We'll spend most of the rest of this chapter discussing each in depth with specific techniques and examples. We'll end the chapter by discussing Bastille Linux, a handy tool with which Red Hat and Mandrake Linux users can automate much of the hardening process.

3.1.1 Installing/Running Only Necessary Software

This is the most obvious of our submaxims/corollaries. But what does "necessary" really mean? What if you don't know whether a given software package is necessary, especially if it was automatically installed when you set up the system?

You have three allies in determining each package's appropriateness:

  • Common sense

  • man

  • Your Linux distribution's package manager (rpm on Red Hat and its derivatives, dpkg and dselect on Debian, and both yast and rpm on SuSE systems).

Common sense, for example, dictates that a firewall shouldn't be running apache and that a public FTP server doesn't need a C compiler. Remember, since our guiding principle is "that which is not expressly permitted must be denied," it follows that "that which is not necessary should be considered needlessly risky."

Division of Labor Between Servers

Put different services on different hosts whenever possible. The more roles a single host plays, the more applications you will need to run on it, and therefore the greater the odds that that particular machine will be compromised.

For example, if a DMZ network contains a web server running Apache, an FTP server running wuftpd, and an SMTP gateway running postfix, a new vulnerability in wuftpd will directly threaten the FTP server, but only indirectly threaten the other two systems. (If compromised, the FTP server may be used to attack them, but the attacker won't be able to capitalize on the same vulnerability she exploited on the FTP server).

If that DMZ contains a single host running all three services, the wuftpd vulnerability will, if exploited, directly impact not only FTP functionality, but also World Wide Web services and Internet email relaying.

If you must combine roles on a single system, aim for consistency.For example, have one host support public WWW services along with public FTP services, since both are used for anonymous filesharing, and have another host provide DNS and SMTP since both are "infrastructure" services. A little division of labor is better than none.

In any case, I strongly recommend against using your firewall as anything but a firewall.

If you don't know what a given command or package does, the simplest way to find out is via a man lookup. All manpages begin with a synopsis of the described command's function. I regularly use manpage lookups both to identify unfamiliar programs and to refresh my memory on things I don't use but have a vague recollection of being necessary.

If there's no manpage for the command/package (or you don't know the name of any command associated with the package), try apropos <string> for a list of related manpages. If that fails, your package manager should, at the very least, be able to tell you what other packages, if any, depend on it. Even if this doesn't tell you what the package does, it may tell you whether it's necessary.

For example, in reviewing the packages on my Red Hat system, suppose I see libglade installed but am not sure I need it. As it happens, there's no manpage for libglade, but I can ask rpm whether any other packages depend on it (Example 3-1).

Example 3-1. Using man, apropos, and rpm to identify a package
[mick@woofgang]$ man libglade

No manual entry for libglade

   

[mick@woofgang]$ apropos libglade

libglade: nothing appropriate

   

[mick@woofgang]$ rpm -q --whatrequires libglade

memprof-0.3.0-8

rep-gtk-gnome-0.13-3

Aha...libglade is part of GNOME. If the system in question is a server, it probably doesn't need the X Window System at all, let alone a fancy frontend like GNOME, so I can safely uninstall libglade (along with the rest of GNOME).

SuSE also has the rpm command, so Example 3-1 is equally applicable to it. Alternatively, you can invoke yast, navigate to Package Management figs/U2192.gif Change/Create Configuration, flag libglade for deletion, and press F5 to see a list of any dependencies that will be affected if you delete libglade.

Under Debian, dpkg has no simple means of tracing dependencies, but dselect handles them with aplomb. When you select a package for deletion (by marking it with a minus sign), dselect automatically lists the packages that depend on it, conveniently marking them for deletion too. To undo your original deletion flag, type "X"; to continue (accepting dselect's suggested additional package deletions), hit RETURN.

3.1.1.1 Commonly unnecessary packages

I highly recommend you not install the X Window System on publicly accessible servers. Server applications (Apache, ProFTPD, and Sendmail, to name a few) almost never require X; it's extremely doubtful that your bastion hosts really need X for their core functions. If a server is to run "headless" (without a monitor and thus administered remotely), then it certainly doesn't need a full X installation with GNOME, KDE, etc., and probably doesn't need even a minimal one.

During Linux installation, deselecting X Window packages, especially the base packages, will return errors concerning "failed dependencies." You may be surprised at just how many applications make up a typical X installation. In all likelihood, you can safely deselect all of these applications, in addition to X itself.

When in doubt, identify and install the package as described previously (and as much X as it needs ? skip the fancy window managers) only if you're positive you need it. If things don't work properly as a result of omitting a questionable package, you can always install the omitted packages later.

Besides the X Window System and its associated window managers and applications, another entire category of applications inappropriate for Internet-connected systems is the software-development environment. To many Linux users, it feels strange to install Linux without also installing GCC, GNU Make, and at least enough other development tools with which to compile a kernel. But if you can build things on an Internet-connected server, so may a successful attacker.

One of the first things any accomplished system cracker does upon compromising a system is to build a "rootkit," a set of standard Unix utilities such as ls, ps, netstat, and top, which appear to behave just like the system's native utilities. Rootkit utilities, however, are designed not to show directories, files, and connections related to the attacker's activities, making it much easier for said activities to go unnoticed. A working development environment on the target system makes it much easier for the attacker to build a rootkit that's optimized for your system.

Of course, the attacker can still upload his own compiler or precompiled binaries of his rootkit tools. Hopefully, you're running Tripwire or some other system-integrity-checker, which will alert you to changes in important system files (see Chapter 11). Still, trusted internal systems, not exposed public systems, should be used for developing and building applications; the danger of making your bastion host "soft and chewy on the inside" (easy to abuse if compromised) is far greater than any convenience you'll gain from doing your builds on it.

Similarly, there's one more type of application I recommend keeping off of your bastion hosts: network monitoring and scanning tools. This is should be obvious, but tcpdump, nmap, nessus, and other tools we commonly use to validate system/network security have tremendous potential for misuse.

As with development tools, security-scanning tools are infinitely more useful to illegitimate users in this context than they are to you. If you want to scan the hosts in your DMZ network periodically (which is a useful way to "check your work"), invest a few hundred dollars in a used laptop system, which you can connect to and disconnect from the DMZ as needed.

While any unneeded service should be either deleted or disabled, the following deserve particular attention:

rpc services

Sun's Remote Procedure Control protocol (which is included nowadays on virtually all flavors of Unix) lets you centralize user accounts across multiple systems, mount remote volumes, and execute remote commands. But RPC isn't a very secure protocol, and you shouldn't be running these types of services on a DMZ hosts anyhow.

Disable (rename) the nfsd and nfsclientd scripts in all subdirectories of /etc/rc.d in which they appear.

Local processes sometimes require the RPC " portmapper," a.k.a. rpcbind. Disable this with care, and try re-enabling it if other things stop working, unless those things are all X-related. (You shouldn't be running X on any publicly available server.)

r-services

rsh, rlogin, and rcp allow remote shell sessions and file transfers using some combination of username/password and source-IP-address authentication. But authentication data is passed in the clear and IP addresses can be spoofed, so these applications are not suitable for DMZ use. If you need their functionality, use Secure Shell (SSH), which was specifically designed as a replacement for the r-services. SSH is covered in detail in Chapter 4.

Comment out the lines corresponding to any "r-commands" in /etc/inetd.conf.

inetd:

The Internet Daemon is a handy way to use a single process (i.e., inetd) to listen on multiple ports and invoke the services on whose behalf it's listening as needed. On a bastion host, however, most of your important services should be invoked as persistent daemons: an FTP server, for example, really has no reason not to run FTPD processes all the time.

Furthermore, most of the services enabled by default in inetd.conf are unnecessary, insecure, or both. If you must use inetd, edit /etc/inetd.conf to disable all services you don't need (or never heard of!). Many of the rpc services I warned against earlier are started in inetd.conf.

linuxconfd

While there aren't any known exploitable bugs in the current version of linuxconf (a system administration tool that can be accessed remotely), its presence is a dead giveaway that you're running Linux (and probably either Red Hat or Mandrake): CERT reports that this service is commonly scanned for and may be used by attackers to identify systems with other vulnerabilities (CERT Current Scanning Activity page 07/08/2002, http://www.cert.org/current/scanning.html).

sendmail

Many people think that sendmail, which is enabled by default on most versions of Unix, should run continuously as a daemon, even on hosts that send email only to themselves (e.g., administrative messages such as crontab output sent to root by the crontab daemon). This is not so: sendmail (or postfix, qmail, etc.) should be run as a daemon only on servers that must receive mail from other hosts. (On other servers, run sendmail to send mail only as needed; you can also execute sendmail -q as a cron job to attempt delivery of queued messages periodically.) Sendmail is usually started in /etc/rc.d/rc2.d or /etc/rc.d/rc3.d.

Telnet, FTP, and POP

These three protocols have one unfortunate characteristic in common: they require users to enter a username and password, which are sent in clear text over the network. Telnet and FTP are easily replaced with ssh and its file-transfer utilities scp and sftp; email can either be automatically forwarded to a different host, left on the DMZ host and read through a ssh session, or downloaded via POP using a "local forward" to ssh (i.e., piped through an encrypted Secure Shell session). All three of these services are usually invoked by inetd.

Remember, one of our operating assumptions in the DMZ is that hosts therein are much more likely to be compromised than internal hosts. When installing software, you should maintain a strict policy of "that which isn't necessary may be used against me." Furthermore, consider not only whether you need a given application but also whether the host on which you're about to install it is truly the best place to run it (see "Division of Labor Between Servers," earlier in this chapter).

3.1.1.2 Disabling services without uninstalling them

Perhaps there are certain software packages you want installed but don't need right away. Or perhaps other things you're running depend on a given package that has a nonessential daemon you wish to disable.

If you run Red Hat or one of its derivatives (Mandrake, Yellow Dog, etc.), you should use chkconfig to manage startup services. chkconfig is a simple tool (Example 3-2).

Example 3-2. chkconfig usage message
[mick@woofgang mick]# chkconfig --help

chkconfig version 1.2.16 - Copyright (C) 1997-2000 Red Hat, Inc.

This may be freely redistributed under the terms of the GNU Public License.

   

usage:   chkconfig --list [name]

         chkconfig --add <name>

         chkconfig --del <name>

         chkconfig [--level <levels>] <name> <on|off|reset>)

To list all the startup services on my Red Hat system, I simply enter chkconfig --list. For each script in /etc/rc.d, chkconfig will list that script's startup status (on or off) at each runlevel. The output of Example 3-3 has been truncated for readability:

Example 3-3. Listing all startup scripts' configuration
[root@woofgang root]# chkconfig --list 

anacron         0:off   1:off   2:on    3:on    4:on    5:on    6:off

httpd           0:off   1:off   2:off   3:off   4:off   5:off   6:off

syslog          0:off   1:off   2:on    3:on    4:on    5:on    6:off

crond           0:off   1:off   2:on    3:on    4:on    5:on    6:off

network         0:off   1:off   2:on    3:on    4:on    5:on    6:off

linuxconf       0:off   1:off   2:on    3:off   4:off   5:off   6:off

(etc.) 

To disable linuxconf in runlevel 2, I'd execute the commands shown in Example 3-4.

Example 3-4. Disabling a service with chkconfig
[root@woofgang root]# chkconfig --level 2 linuxconf off

[root@woofgang root]# chkconfig --list linuxconf

linuxconf       0:off   1:off   2:off   3:off   4:off   5:off   6:off

(The second command, chkconfig --list linuxconf, is optional but useful in showing the results of the first.)

On SuSE systems, edit the startup script itself (the one in /etc/init.d), and then run the command insserv (no flags or arguments necessary) to change automatically the symbolic links that determine the runlevels in which it's started. Each SuSE startup script begins with a header, comprised of comment lines, which dictate how init should treat it (Example 3-5).

Example 3-5. A SuSE INIT INFO header
# /etc/init.d/lpd

#

### BEGIN INIT INFO

# Provides: lpd

# Required-Start: network route syslog named

# Required-Stop: network route syslog

# Default-Start: 2 3 5

# Default-Stop:

# Description:  print spooling service

### END INIT INFO

For our purposes, the relevant settings are Default-Start, which lists the runlevels in which the script should be started, and Default-Stop, which lists the runlevels in which the script should be stopped. Actually, since any script started in runlevel 2, 3, or 5 is automatically stopped when that runlevel is exited, Default-Stop is often left empty.

Any time you change a startup script's INIT INFO header on a SuSE system, you must then run the command insserv to tell SuSE to change the start/stop links accordingly (in /etc/init.d's "rc" subdirectories). insserv is run without arguments or flags.

For more information about the SuSE's particular version of the System V init-script system, see SuSE's init.d(7) manpage.

On all other Linux distributions, you can disable a service simply by deleting or renaming its links in the appropriate runlevel directories under /etc/rc.d/. For example, if you're configuring a web server that doesn't need to be its own DNS server, you probably want to disable BIND. The easiest way to do this without deleting anything is by renaming all links to /etc/init.d/ (Example 3-6).

Example 3-6. Disabling a startup script by renaming its symbolic links
[root@woofgang root]# mv /etc/rc.d/rc2.d/S30named /etc/rc.d/rc2.d/disabled_S30named

[root@woofgang root]# mv /etc/rc.d/rc3.d/S30named /etc/rc.d/rc3.d/disabled_S30named

[root@woofgang root]# mv /etc/rc.d/rc5.d/S30named /etc/rc.d/rc5.d/disabled_S30named

(Note that your named startup script may have a different name and exist in different or additional subdirectories of /etc/rc.d.)

3.1.2 Keeping Software Up to Date

It isn't enough to weed out unnecessary software: all software that remains, including both the operating system itself and "user-space" applications, must be kept up to date. This is a more subtle problem than you might think, since many Linux distributions offer updates on both a package-by-package basis (e.g., the Red Hat Errata web site) and in the form of new distribution revisions (e.g., new CD-ROM sets).

What, then, constitutes "up to date"? Does it mean you must immediately upgrade your entire system every time your distribution of choice releases a new set of CD-ROMs? Or is it okay simply to check the distribution's web page every six months or so? In my opinion, neither is a good approach. (Not that these are the only two choices; they represent extremes.)

3.1.2.1 Distribution (global) updates versus per-package updates

The good news is that it's seldom necessary to upgrade a system completely just because the distribution on which it's based has undergone an incremental revision (e.g., 7.2 figs/U2192.gif 7.3). The bad news is that updates to individual packages should probably be applied much more frequently than that: if you have one or more Internet-connected systems, I strongly recommend you subscribe to your distribution's security-announcement mailing list and apply each relevant security patch as soon as it's announced.

Remember, the people who announce "new" security vulnerabilities as a public service are not always the first to discover them. The prudent assumption for any such vulnerability is that the "bad guys" already know about it and are ready to exploit it if they find it on your systems.

Therefore, I repeat: the only way to minimize your exposure to well-known vulnerabilities is to do the following:

  • Subscribe to your distribution's security-announcement mailing list

  • Apply each security patch immediately after receiving notice of it

  • If no patch is available for an application with widely exploited vulnerabilities, disable that application until a patch is released.

A "global" revision to an entire Linux distribution is not a security event in itself. Linux distributions are revised to add new software packages, reflect new functionality, and provide bug fixes. Security is hopefully enhanced too, but not necessarily. Thus, while there are various reasons to upgrade to a higher numbered revision of your Linux distribution (stability, new features, etc.), doing so won't magically make your system more secure.

In general, it's good practice to stick with a given distribution version for as long as its vendor continues to provide package updates for it, and otherwise to upgrade to a newer (global) version only if it has really compelling new features. In any Linux distribution, an older but still supported version with all current patches applied is usually at least as secure as the newest version with patches and probably more secure than the new version without patches.

In fact, don't assume that the CD-ROM set you just received in the mail directly from SuSE, for example, has no known bugs or security issues just because it's new. You should upgrade even a brand-new operating system (or at least check its distributor's web site for available updates) immediately after installing it.

I do not advocate the practice of checking for vulnerabilities only periodically and not worrying about them in the interim: while better than never checking, this strategy is simply not proactive enough. Prospective attackers won't do you the courtesy of waiting after your quarterly upgrade session before striking. (If they do, then they know an awful lot about your system and will probably get in anyhow!)

Therefore, I strongly recommend you get into the habit of applying security-related patches and upgrades in an ad-hoc manner ? i.e., apply each new patch as soon as it's announced.

Should I Always Update?

Good system administrators make clear distinctions between stable "production" systems and volatile "research and development" (r&d) systems. One big difference is that on production systems, you don't add or remove software arbitrarily. Therefore, you may not feel comfortable applying every update for every software package on your production system as soon as they're announced.

That's probably prudent in many cases, but let me offer a few guidelines:

  • Apply any update addressing a "remote root" vulnerability that could lead to remote users gaining administrative access to the system.

  • If the system supports interactive/shell use by more than a few users (e.g., via Telnet, ssh, etc.), then apply any update addressing an "escalation of local privileges" vulnerability that could allow an unprivileged user to increase their level of privilege.

  • If the system doesn't support interactive/shell use except by one or two administrators, then you can probably postpone updates that address "escalation of privilege" bugfixes.

  • A nonsecurity-related update may be safely skipped, unless, of course, that update is intended to fix some source of system instability. (Attackers often intentionally induce instability in the execution of more complex attacks.)

In my experience, it's relatively rare for a Linux package update to affect system stability negatively. The only exception to this is kernel updates: new major versions are nearly always unstable until the fourth or fifth minor revision (e.g., avoid kernel Version X.Y.0: wait for Version X.Y.4 or X.Y.5).

3.1.2.2 Whither X-based updates?

In subsequent sections of this chapter, I'll describe methods of updating packages in Red Hat, SuSE, and Debian systems. Each of these distributions supports both automated and manual means of updating packages, ranging from simple commands such as rpm -Uvh ./mynewrpm-2.0.3.rpm (which works in all rpm-based distributions: Red Hat, SuSE, etc.) to sophisticated graphical tools such as yast2 (SuSE only).

Given that earlier in this chapter I recommended against installing the X Window System on your bastion hosts, it may seem contradictory for me to cover X-based update utilities. There are two good reasons to do so, however:

  • For whatever reason, you may decide that you can't live without X on one or more of your bastion hosts.

  • Just because you don't run X on a bastion host doesn't mean you can't run an X-based update tool on an internal host, from which you can upload the updated packages to your bastion hosts via a less glamorous tool such as scp (see Chapter 4).

3.1.2.3 How to be notified of and obtain security updates: Red Hat

If you run Red Hat 6.2 or later, the officially recommended method for obtaining and installing updates and bug/security fixes (errata in Red Hat's parlance) is to register with the Red Hat Network and then either schedule automatic updates on the Red Hat Network web site or perform them manually using the command up2date. While all official Red Hat packages may also be downloaded anonymously via FTP and HTTP, Red Hat Network registration is necessary to both schedule automatic notifications and downloads from Red Hat and use up2date.

At first glance, the security of this arrangement is problematic: Red Hat encourages you to remotely store a list with Red Hat of the names and versions of all your system's packages and hardware. This list is transferred via HTTPS and can only be perused by you and the fine professionals at Red Hat. In my opinion, however, the truly security conscious should avoid providing essential system details to strangers.

There is a way around this. If you can live without automatically scheduled updates and customized update lists from Red Hat, you can still use up2date to generate system-specific update lists locally (rather than have them pushed to you by Red Hat). You can then download and install the relevant updates automatically, having registered no more than your email address and system version/architecture with Red Hat Network.

First, to register with the Red Hat Network, execute the command rhn_register . (If you aren't running X, then use the --nox flag, e.g., rhn_register --nox.) In rhn_register's Step 2 screen (Step 1 is simply a license click-though dialogue), you'll be prompted for a username, password, and email address: all three are required. You will then be prompted to provide as little or as much contact information as you care to disclose, but all of it is optional.

In Step 3 (system profile: hardware), you should enter a profile name, but I recommend you uncheck the box next to "Include information about hardware and network." Similarly, in the screen after that, I recommend you uncheck the box next to "Include RPM packages installed on this system in my System Profile." By deselecting these two options, you will prevent your system's hardware, network, and software-package information from being sent to and stored at Red Hat.

Now, when you click the "Next" button to send your profile, nothing but your Red Hat Network username/password and your email address will be registered. You can now use up2date without worrying quite so much about who possesses intimate details about your system.

Note there's one more useful Red Hat Network feature you'll subsequently miss: automatic, customized security emails. Therefore, be sure to subscribe to the Redhat-Watch-list mailing list using the online form at https://listman.redhat.com. This way, you'll receive emails concerning all Red Hat bug and security notices (i.e., for all software packages in all supported versions of Red Hat), but since only official Red Hat notices may be posted to the list, you needn't worry about Red Hat swamping you with email. If you're worried anyhow, a "daily digest" format is available (in which all the day's postings are sent to you in a single message).

Once you've registered with the Red Hat Network via rhn_register (regardless of whether you opt to send hardware/package info), you can run up2date. First, you need to configure up2date, but this task has its own command, up2date-config (Figure 3-1). By default, both up2date and up2date-config use X-Windows; but like rhn_register, both support the --nox flag if you prefer to run them from a text console.

Figure 3-1. up2date-config
figs/bssl_0301.gif

up2date-config is fairly self-explanatory, and you should need to run it only once (though you may run it at any time). A couple of settings, though, are worth noting. First is whether up2date should verify each package's cryptographic signature with gpg. I highly recommend you use this feature (it's selected by default), as it reduces the odds that up2date will install any package that has been corrupted or "trojaned" by a clever web site hacker.

Also, if you're downloading updates to a central host from which you plan to "push" (upload) them to other systems, you'll definitely want to select the option "After installation, keep binary packages on disk" and define a "Package storage directory." You may or may not want to select "Do not install packages after retrieval." The equivalents of these settings in up2date's ncurses mode (up2date-config --nox) are keepAfterInstall, storageDir, and retrieveOnly, respectively.

Truth be told, I'm leery of relying on automated update tools very much, even up2date (convenient though it is). Web and FTP sites are hacked all the time, and sooner or later a Linux distributor's site will be compromised and important packages replaced with Trojaned versions.

Therefore, if you use up2date, it's essential you use its gpg functionality as described earlier. One of the great strengths of the rpm package format is its support of embedded digital signatures, but these do you no good unless you verify them (or allow up2date to verify them for you).

The command to check an rpm package's signature manually is rpm --checksig /path/packagename.rpm. Note that both this command and up2date require you to have the package gnupg installed.

Now you can run up2date. up2date will use information stored locally by rhn_register to authenticate your machine to the Red Hat Network, after which it will download a list of (the names/versions of) updates released since the last time you ran up2date. If you specified any packages to skip in up2date-config, up2date won't bother checking for updates to those packages. Figure 3-2 shows a screen from a file server of mine on which I run custom kernels and therefore don't care to download kernel-related rpms.

Figure 3-2. Red Hat's up2date: skipping unwanted updates
figs/bssl_0302.gif

After installing Red Hat, registering with the Red Hat Network, configuring up2date and running it for the first time to make your system completely current, you can take a brief break from updating. That break should last, however, no longer than it takes to receive a new security advisory email from Redhat-Watch that's relevant to your system.

Why Not Trust Red Hat?

I don't really have any reason not to trust the Red Hat Network; it's just that I don't think it should be necessary to trust them. (I'm a big fan of avoiding unnecessary trust relationships!)

Perhaps you feel differently. Maybe the Red Hat Network's customized autoupdate and autonotification features will for you mean the difference between keeping your systems up-to-date and not. If so, then perhaps whatever risk is involved in maintaining a detailed list of your system information with the Red Hat Network is an acceptable one.

In my opinion, however, up2date is convenient and intelligent enough by itself to make even that small risk unnecessary. Perhaps I'd think differently if I had 200 Red Hat systems to administer rather than two.

But I suspect I'd be even more worried about remotely caching an entire network's worth of system details. (Plus I'd have to pay Red Hat for the privilege, since each RHN account is allowed only one complimentary system "entitlement"/subscription.) Far better to register one system in the manner described earlier (without sending details) and then use that system to push updates to the other 199, using plain old rsync, ssh, and rpm.

In my experience, the less information you needlessly share, the less that will show up in unwanted or unexpected hands.

3.1.2.4 RPM updates for the extremely cautious

up2date's speed, convenience, and automated signature checking are appealing. On the other hand, there's something to be said for fully manual application of security updates. Updating a small number of packages really isn't much more trouble with plain old rpm than with up2date, and it has the additional benefit of not requiring Red Hat Network registration. Best of all from a security standpoint, what you see is what you get: you don't have to rely on up2date to relay faithfully any and all errors returned in the downloading, signaturechecking, and package-installation steps.

Here, then, is a simple procedure for applying manual updates to systems running Red Hat, Mandrake, SuSE, and other rpm-based distributions:

Download the new package

The security advisory that notified you of the new packages also contains full paths to the update on your distribution's primary FTP site. Change directories to where you want to download updates and start your FTP client of choice. For single-command downloading, you can use wget (which of course requires the wget package), e.g.:

wget -nd --passive-ftp ftp://updates.redhat.com/7.0/en/os/i386/rhs-printfilters-

1.81-4.rh7.0.i386.rpm
Verify the package's gpg signature

You'll need to have the gnupg package installed on your system, and you'll also need your distribution's public package-signing key on your gpg key ring. You can then use rpm to invoke gpg via rpm's --checksig command, e.g.:

rpm --checksig

./rhs-printfilters-1.81-4.rh7.0.i386.rpm
Install the package using rpm's update command (-U)

Personally, I like to see a progress bar, and I also like verbose output (errors, etc.), so I include the -h and -v flags, respectively. Continuing the example of updating rhs-printfilters, the update command would be:

rpm -Uhv

./rhs-printfilters-1.81-4.rh7.0.i386.rpm

Note that in both rpm usages, you may use wildcards or multiple filenames to act on more than one package, e.g.:

rpm --checksig ./perl-*

and then, assuming the signature checks were successful:

rpm -Uhv ./perl-*

3.1.2.5 How to be notified of and obtain security updates: SuSE

As with so much else, automatic updates on SuSE systems can be handled through yast and yast2. Chances are if you run a version of SuSE prior to 8.0, you'll want both of these on your bastion host, since yast2 didn't fully replace yast until SuSE 8.0. Either can be used for software updates, so let's discuss both.

To use yast to automatically update all packages for which new RPM files are available, start yast and select add/remove programsfigs/U2192.gifupgrade entire system. yast will give you the opportunity to either install all new patches automatically or designate which to install and which to skip.

This method takes a long time: depending on which mirror you download your patches from, such an update can last anywhere from one to several hours. In practice, therefore, I recommend using the "upgrade entire system" option immediately after installing SuSE. Afterwards, you'll want to download and install updates individually as they're released by using plain old rpm (e.g., rpm -Uvh ./mynewpackage.rpm).

The best way to keep on top of new security updates is to subscribe to the official SuSE security-announcement mailing list, suse-security-announce. To subscribe, use the online form at http://www.suse.com/en/support/mailinglists/index.html.

Whenever you receive notice that one of the packages on your system has a vulnerability addressed by a new patch, follow the instructions in the notice to download the new package, verify its GNUpg signature (as of SuSE Linux version 7.1, all SuSE RPMs are signed with the key build@suse.com), and install it. This procedure is essentially the same as that described earlier in the section "RPM updates for the extremely cautious."

Checking Package Versions

To see a list of all currently installed packages and their version numbers on your RPM-based system, use this command:

rpm -qa

To see if a specific package is installed, pipe this command to grep, specifying part or all of the package's name. For example:

rpm -qa |grep squid

on my SuSE 7.1 system returns this output:

squid23-2.3.STABLE4-75

The equivalent commands for deb-package-based distributions like Debian would be dpkg -l and dpkg -l |grep squid, respectively. Of course, either command can be redirected to a file for later reference (or off-system archival ? e.g., for crash or compromise recovery) like this:

rpm -qa > packages_07092002.txt

3.1.2.6 SuSE's online-update feature

In addition to yast and rpm, you can also use yast2 to update SuSE packages. This method is particularly useful for performing a batch update of your entire system after installing SuSE. yast2 uses X by default, but will automatically run in ncurses mode (i.e., with an ASCII interface structured identically to the X interface) if the environment variable DISPLAY isn't set.

In yast2, start the "Software" applet, and select "Online Update." You have the choice of either an automatic update in which all new patches are identified, downloaded, and installed or a manual update in which you're given the choice of which new patches should be downloaded and installed (Figure 3-3). In either option, you can click the "Expert" button to specify an FTP server other than ftp.suse.com.

Figure 3-3. Selecting patches in yast2
figs/bssl_0303.gif

Overall, yast2's Online Update functionality is simple and fast. The only error I've encountered running it on my two SuSE servers was the result of invoking yast2 from an xterm as an unprivileged user: yast2 claimed that it couldn't find the update list on ftp.suse.com, which wasn't exactly true. The real problem was that yast2 couldn't write that file locally where it needed to because it was running with my non-root privileges.

Invoking yast2 from a window-manager menu (in any window manager that susewm configures) obviates this problem: you will be prompted for the root password if you aren't running X as root. Running X as root, of course, is another workaround, but not one I recommend due to the overall insecurity of X-Windows. A better approach is to open a terminal window and issue these commands (output omitted):

bash-$ su

bash-# export DISPLAY=""

bash-# yast2

Setting the environment variable DISPLAY to null in this way (make sure not to put any whitespace between the quotation marks) will force yast2 to run in your terminal window in ncurses mode; it won't matter which user started the underlying X session.

3.1.2.7 How to be notified of and obtain security updates: Debian

As is typical of Debian GNU/Linux, updating Debian packages is less flashy yet simpler than with most other distributions. The process consists mainly of two commands (actually, one command, apt-get, invoked twice but with different options):

apt-get update

apt-get -u upgrade

The first command, apt-get update, updates your locally cached lists of available packages (which are stored, if you're curious, in /var/state/apt/lists). This is necessary for apt-get to determine which of your currently installed packages have been updated.

The second command, apt-get -u upgrade, causes apt-get to actually fetch and install the new versions of your local outdated packages. Note that as with most other Linux package formats, the deb format includes pre- and post-installation scripts; therefore, it isn't necessarily a good idea to run an apt-get upgrade unattended, since one or more scripts may prompt you for configuration information.

That's really all there is to it! Naturally, errors are possible: a common cause is outdated FTP/HTTP links in /etc/apt/sources.list. If apt-get seems to take too long to fetch package lists and/or reports that it can't find files, try deleting or replacing the sources.list entry corresponding to the server that apt-get was querying before it returned the error. For a current list of Debian download sites worldwide, see http://www.debian.org/distrib/ftplist.

Another common error is new dependencies (ones that didn't apply when you originally installed a given package), which will cause apt-get to skip the affected package. This is fixed by simply invoking apt-get again, this time telling it to install the package plus any others on which it depends.

For example, suppose that in the course of an upgrade session, apt-get reports that it's skipping the package blozzo. After apt-get finishes the rest of the upgrade session, you enter the command:

apt-get install blozzo

apt-get will then attempt to install the latest version of blozzo and will additionally do a more thorough job of trying to resolve its dependencies. If your old version of blozzo is hopelessly obsolete, however, it may be necessary to upgrade your entire distribution; this is done with the command apt-get -u dist-upgrade.

Detailed instructions on using apt-get can be found in the apt-get(8) manpage, as well as in the APT HOWTO (available at http://www.debian.org/doc/manuals/apt-howto).

To receive prompt, official notification of Debian security fixes, subscribe to the debian-security-announce email list. An online subscription form is available at http://www.debian.org/MailingLists/subscribe.

Unfortunately, the deb package format doesn't currently support GNUpg signatures, or even md5 hashes, nor are external hashes or GNUpg signatures maintained or checked. Therefore, be careful to stick to official Debian FTP mirror sites when using apt-get.

Reportedly, a future version of the deb package format will support GNUpg signatures.

3.1.3 Deleting Unnecessary User Accounts and Restricting Shell Access

One of the popular distributions' more annoying quirks is the inclusion of a long list of entries in /etc/passwd for application-specific user accounts, regardless of whether those applications are even installed. (For example, my SuSE 7.1 system created 48 entries during installation!) While few of these are privileged accounts, many can be used for interactive login (i.e., they specify a real shell rather than /bin/false). This is not unique to SuSE: my Red Hat 7.0 system created 33 accounts during installation, and my Debian 2.2 system installed 26.

While it's by no means certain that a given unused account can and will be targeted by attackers, I personally prefer to err on the side of caution, even if that makes me look superstitious in some peoples' eyes. Therefore, I recommend that you check /etc/passwd and comment out any unnecessary entries.

If you aren't sure what a given account is used for but see that account has an actual shell specified, one way to determine whether an account is active is to see whether it owns any files, and if so, when they were last modified. This is easily achieved using the find command.

Suppose I have a recently installed web server whose /etc/passwd file contains, among many others, the following entry:

yard:x:29:29:YARD Database Admin:/usr/lib/YARD:/bin/bash

I have no idea what the YARD database might be used for. Manpage lookups and rpm queries suggest that it isn't even installed. Still, before I comment out yard's entry in /etc/passwd, I want to make sure the account isn't active. It's time to try find / -user and ls -lu (Example 3-7).

Example 3-7. Using find with the -user flag
root@woofgang:~ # find / -user yard -print

/usr/lib/YARD

   

root@woofgang:~ # ls -lu /usr/lib/YARD/

total 20

drwxr-xr-x    2 yard     yard           35 Jan 17  2001 .

drwxr-xr-x   59 root     root        13878 Dec 13 18:31 ..

As we see in Example 3-7, yard owns only one directory, /usr/lib/YARD, and it's empty. Furthermore, according to ls -lu (which displays and lists files by access times), the directory hasn't been accessed since January 17. Since the system was installed in October, this date must refer to the directory's creation on my installation media by SuSE! Clearly, I can safely assume that this account isn't in use.

Some accounts that are usually necessary if present are as follows:

  • root

  • bin

  • daemon

  • halt

  • shutdown

  • man

  • at

Some accounts that are often unnecessary, at least on bastion hosts, are as follows:

  • uucp

  • games

  • gdm

  • xfs

  • rpcuser

  • rpc

If nothing else, you should change the final field (default shell), in unknown or process-specific accounts' entries in /etc/passwd, from a real shell to /bin/false ? only accounts used by human beings should need shells.

3.1.4 Restricting Access to Known Users

Some FTP daemons allow anonymous login by default. If your FTP server is intended to provide public FTP services, that's fine; but if it isn't, then there's no good reason to leave anonymous FTP enabled.

The same goes for any other service running on a publicly accessible system: if that service supports but doesn't actually require anonymous connections, then the service should be configured to accept connections only from authenticated, valid users. Restricting access to FTP, HTTP, and other services is described in subsequent chapters.

3.1.5 Running Services in chrooted Filesystems

One of our most important threat models is that of the hijacked daemon: if a malicious user manages to take over and effectively "become" a process on our system, he will assume the privileges on our system that that process has. Naturally, developers are always on the alert for vulnerabilities, such as buffer overflows, that compromise their applications, which is why you must keep on top of your distribution's security advisories and package updates.

However, it's equally important to mitigate the risk of potential daemon vulnerabilities, i.e., vulnerabilities that might be unknown to anyone but the "bad guys." There are two primary means of doing so: running the process with as low a set of privileges as possible (see the next section) and running the process in a chroot jail.

Normally, a process can see and interact with as much of a system's filesystem as the user account under which the process runs. Since most of the typical Linux host's filesystem is world-readable, that amounts to a lot of real estate. The chroot system call functionally transposes a process into a subset of the filesystem, effectively redefining the / directory for that process to a small subdirectory under the real root.

For example, suppose a system has the following filesystem hierarchy (see Figure 3-4).

Figure 3-4. Example network architecture
figs/bssl_0304.gif

For most processes and users, configuration files are found in /etc, commands are found in /usr/bin, and various "volatile" files such as logs are found in /var. However, we don't want our DNS daemon, named, to "see" the entire filesystem, so we run it chrooted to /var/named. Thus, from named's perspective, /var/named/etc is /etc, /var/named/usr/bin is /usr/bin, and /var/named/var appears as /var. This isn't a foolproof method of containment, but it helps.

Many important network daemons now support command-line flags and other built-in means of being run chrooted. Subsequent chapters on these daemons describe in detail how to use this functionality.

(Actually, almost any process can be run chrooted if invoked via the chroot command, but this usually requires a much more involved chroot jail than do commands with built-in chroot functionality. Most applications are compiled to use shared libraries and won't work