11.3 Using the Standard Unix Randomness Infrastructure

11.3.1 Problem

You want to use random numbers on a modern-day Unix machine.

11.3.2 Solution

On most modern Unix systems, there are two devices from which you can read: /dev/random, which is expected to produce entropy, and /dev/urandom, which is expected to provide cryptographically secure pseudo-random values. In reality, these expectations may not always be met, but in practice, it seems reasonably safe to assume that they are.

We strongly recommend accessing these devices through the API we present in Recipe 11.2.

11.3.3 Discussion

If you need a cryptographically strong random number source that is nonetheless reproducible, /dev/random will not suit your purposes. Use one of the other PRNGs discussed in this chapter.

Most modern Unix operating systems have two devices that produce random numbers: /dev/random and /dev/urandom. In theory, /dev/random may block and should produce data that is statistically close to pure entropy, while /dev/urandom should return immediately, providing only cryptographic randomness.

The real world is somewhat messy, though. First, your application may need to run on a system that does not have these devices. (In that case, see Recipe 11.19, where we discuss solutions to this problem.[1]) Any reasonable version of Linux, FreeBSD, OpenBSD, or NetBSD will have these devices. They are also present on Mac OS X 10.1 or later, Solaris 9 or later, AIX 5.2 or later, HP-UX 11i or later, and IRIX 6.5.19 or later. As of this writing, only dead or officially "about to die" Unix variants, such as Tru64 and Ultrix, lack these devices. Note that each operating system tends to have its own implementation of these devices. We haven't looked at them all, so we cannot, in general, vouch for how strong and efficient these generators are, but we don't think you should worry about this issue in practice. (There are almost always bigger fish to fry.)

[1] If you want to interoperate with such platforms (there are still plenty of systems without /dev/random and /dev/urandom), that reinforces the utility of using our API; simply link against code that implements our API using the solution from Recipe 11.8 instead of the solution from this recipe.

Second, depending on the operating system, the entropy produced by /dev/random may be reused by /dev/urandom. While few (if any) Unix platforms try to guarantee a clean separation of entropy, this is more of a theoretical problem than a practical problem; it is not something about which we personally would worry. Conversely, depending on the operating system, use of /dev/urandom can drain entropy, denying service to the /dev/random device.

Finally, most operating systems do not actually guarantee that /dev/urandom is properly seeded. To understand why, you need to know something about what generally goes on under the hood. Basically, the randomness infrastructure tries to cull randomness from user input. For example, tiny bits of entropy can be derived from the time between console keystrokes. Unfortunately, the system may start up with very little entropy, particularly if the system boots without user intervention.

To avoid this problem, most cryptographic pseudo-random number generators stash away output before the system shuts down, which is used as a seed for the pseudo-random number generator when it starts back up. If the system can reboot without the seed being compromised (a reasonable assumption unless physical attacks are in your threat model, in which case you have to mitigate risk at the physical level), /dev/urandom will produce good results.

The only time to get really paranoid about a lack of entropy is before you are sure the infrastructure has been seeded well. In particular, a freshly installed system may not have any entropy at all. Many people choose to ignore such a threat, and it is reasonable to do so because it is a problem that the operating system should be responsible for fixing.

However, if you want to deal with this problem yourself, be aware that all of the operating systems that have a /dev/random device (as far as we can determine) monitor all keyboard events, adding those events to their internal collection of entropy. Therefore, you can use code such as that presented in Recipe 11.20 to gather sufficient entropy from the keyboard, then immediately throw it away (because the operating system will also be collecting it). Alternatively, you can collect entropy yourself using the techniques discussed in Recipe 11.22 and Recipe 11.23, then run your own cryptographic pseudo-random number generator (see Recipe 11.5).

The /dev/random and /dev/urandom devices behave just like files. You should read from these devices by opening the files and reading data from them. There are a few common "gotchas" when using that approach, however. First, the call to read data may fail. If you do not check for failure, you may think you got a random number when, in reality, you did not.

Second, people will occasionally use the API functions improperly. In particular, we have seen people who assume that the read( ) or fread( ) functions return a value or a pointer to data. Instead, they return -1 on failure, and otherwise return the number of bytes read.

When using standard C runtime functions, we recommend using read( ). If you are reading from /dev/urandom, read( ) will successfully return unless a signal is delivered during the call (in which case the call should be made again), the operating system is misconfigured, or there is some other catastrophic error. Therefore, if read( ) is unsuccessful, retry when the value of errno is EINTR, and fail unconditionally otherwise. You should also check that the return value is equal to the number of bytes you requested to read, because some implementations may limit the amount of data you can read at once from this device. If you get a short read, merely continue to read until you collect enough data.

When using /dev/random, things are the same if you are performing regular blocking reads. Of course, if not enough entropy is available, the call will hang until the requested data is available or until a signal interrupts the call.

If you don't like that behavior, you can make the file descriptor nonblocking, meaning that the function will return an error and set errno to EAGAIN if there isn't enough data to complete the entire read. Note that if some (but not all) of the requested data is ready, it will be returned instead of giving an error. In that case, the return value of read( ) will be smaller than the requested amount.

Given an integer file descriptor, the following code makes the associated descriptor nonblocking:

#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
   
void spc_make_fd_nonblocking(int fd) {
  int flags;
   
  flags = fcntl(fd, F_GETFL);  /* Get flags associated with the descriptor. */
  if (flags =  = -1) {
    perror("spc_make_fd_nonblocking failed on F_GETFL");
    exit(-1);
  }
  flags |= O_NONBLOCK;         
  /* Now the flags will be the same as before, except with O_NONBLOCK set.
   */
  if (fcntl(fd, F_SETFL, flags) =  = -1) {   
    perror("spc_make_fd_nonblocking failed on F_SETFL");
    exit(-1);
  }
}

Here, we will demonstrate how to use /dev/random and /dev/urandom properly by binding them to the API we developed in Recipe 11.2. We will implement spc_entropy( ) by reading from /dev/random in nonblocking mode. We will implement spc_rand( ) by reading from /dev/urandom. Finally, we will implement spc_keygen( ) by reading as much data as possible from /dev/random in a nonblocking fashion, then falling back to /dev/urandom when /dev/random is dry.

Note that we need to open /dev/random on two file descriptors, one blocking and one not, so that we may avoid race conditions where spc_keygen( ) expects a function to be nonblocking but spc_entropy( ) has set the descriptor to blocking in another thread.

In addition, we assume that the system has sufficient entropy to seed /dev/urandom properly and /dev/random's entropy is not reused by /dev/urandom. If you are worried about either of these assumptions, see the recipes suggested earlier for remedies.

Note that you can expect that /dev/random output is properly postprocessed (whitened) to remove any patterns that might facilitate analysis in the case that the data contains less entropy than expected.

This code depends on the spc_make_fd_nonblocking( ) function presented earlier.

#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <fcntl.h>
#include <errno.h>
   
static int spc_devrand_fd           = -1,
           spc_devrand_fd_noblock =   -1, 
           spc_devurand_fd          = -1;
   
void spc_rand_init(void) {
  spc_devrand_fd         = open("/dev/random",  O_RDONLY);
  spc_devrand_fd_noblock = open("/dev/random",  O_RDONLY);
  spc_devurand_fd        = open("/dev/urandom", O_RDONLY);
   
  if (spc_devrand_fd =  = -1 || spc_devrand_fd_noblock =  = -1) {
    perror("spc_rand_init failed to open /dev/random");
    exit(-1);
  }
  if (spc_devurand_fd =  = -1) {
    perror("spc_rand_init failed to open /dev/urandom");
    exit(-1);
  }
  spc_make_fd_nonblocking(spc_devrand_fd_noblock);
}
   
unsigned char *spc_rand(unsigned char *buf, size_t nbytes) {
  ssize_t       r;
  unsigned char *where = buf;
   
  if (spc_devrand_fd =  = -1 && spc_devrand_fd_noblock =  = -1 && spc_devurand_fd =  = -1)
    spc_rand_init(  );
  while (nbytes) {
    if ((r = read(spc_devurand_fd, where, nbytes)) =  = -1) {
      if (errno =  = EINTR) continue;
      perror("spc_rand could not read from /dev/urandom");
      exit(-1);
    }
    where  += r;
    nbytes -= r;
  }
  return buf;
}
   
unsigned char *spc_keygen(unsigned char *buf, size_t nbytes) {
  ssize_t       r;
  unsigned char *where = buf;
   
  if (spc_devrand_fd =  = -1 && spc_devrand_fd_noblock =  = -1 && spc_devurand_fd =  = -1)
    spc_rand_init(  );
  while (nbytes) {
    if ((r = read(spc_devrand_fd_noblock, where, nbytes)) =  = -1) {
      if (errno =  = EINTR) continue;
      if (errno =  = EAGAIN) break;
      perror("spc_rand could not read from /dev/random");
      exit(-1);
    }
    where  += r;
    nbytes -= r;
  }
  spc_rand(where, nbytes);
  return buf;
}
   
unsigned char *spc_entropy(unsigned char *buf, size_t nbytes) {
  ssize_t       r;
  unsigned char *where = buf;
   
  if (spc_devrand_fd =  = -1 && spc_devrand_fd_noblock =  = -1 && spc_devurand_fd =  = -1)
    spc_rand_init(  );
  while (nbytes) {
    if ((r = read(spc_devrand_fd, (void *)where, nbytes)) =  = -1) {
      if (errno =  = EINTR) continue;
      perror("spc_rand could not read from /dev/random");
      exit(-1);
    }
    where  += r;
    nbytes -= r;
  }
  return buf;
}

11.3.4 See Also

Recipe 11.2, Recipe 11.5, Recipe 11.8, Recipe 11.19, Recipe 11.20, Recipe 11.22, Recipe 11.23