Hack 59 Mirroring Files and Directories with rsync


With rsync, supplied by default on OS X, quick mirrors and backups are a command line away.

As its name suggests, rsync synchronizes files and folders from one location to another. That location could be another directory on your current machine, or any other rsync-enabled machine that you have access to. Using the rsync protocol, only differences between files are transferred; if you're transferring a 30MB text file that had only a few spelling corrections, you'll transfer only those corrections, not the entire 30MB. You won't believe how much of a time-saver this is until you turn your head back and realize it's finished.

I've always been fond of the learning-by-doing school of thought, so open up a Terminal and enter the following command:

% rsync -vaz ~/Library ~/Backups

Let's break that down: rsync is the name of our utility and -vaz are some flags we've passed to it. By issuing an rsync --help at the command line, we quickly find out what that means:

-v, --verbose  increase verbosity
-a, --archive  archive mode
-z, --compress compress file data

Archive mode is a special rsync configuration that is suitable for mirroring. As the manual suggests (see man rsync), archive mode is a quick way of saying you want recursion and to preserve almost everything. In this case, everything means permissions, ownership, file modification and access times, and so on. The -z flag, for compression, is more useful when you're handling mirrors or backups over a network (see later in this hack); it'll compress the data before sending it over your connection, further decreasing the amount of time the command will take.

The second part of the rsync command line is what we want to back up or mirror. In this case, we're saying "hey, take my entire Library directory and back it up to the third part of the rsync command," which is a location in our home directory called Backups. We can back up anything on our drive to anywhere else on our drive. The following command, for instance, backs up the movies in my home directory to another mounted hard drive (named MouthWash):

% rsync -vaz ~/Movies /Volumes/MouthWash

As mentioned, the best part of rsync is the protocol; once you do the initial backup, all future backups will be immensely faster than normal, since only the changed data will be acted upon. This is especially important when you start thinking about rsync over the network, like the following:

% rsync -vaz ~/Library

This is almost the same as the first command, only this time we've added an IP address and a colon to our place to back up the files to. With this command (and rsync installed on the machine at, we'll be backing up the files over the network to a different machine entirely. If one day I delete my ~/Library/ directory accidentally, all I need to do is reverse the procedure:

% rsync -vaz ~/Library

which takes all the files located in the Backup directory at and sticks them into my Library directory on this machine. Thankfully, we can exclude certain files from a backup or restoration. In the following example, we won't back up any file or directory that has the word Cache in it:

% rsync -vaz --exclude=*Cache* ~/Library

You can use as many excludes as necessary; here, we're stopping caches as well as our Mail directories:

% rsync -vaz --exclude=Mail/ --exclude=*Cache* ~/Library

By default, an rsync command is nondestructive, meaning that it will only add files to a mirror or backup, never remove them. This can be unwanted at times, as you don't want old files that you've deleted to continue to exist at the backup location. That's where --delete comes in:

% rsync -az --delete --stats ~/Library ~/Backup

In this case, we've removed the verbose option, added --delete, which will remove files in Backup that no longer exist in Library, and added a nice little ending report with the --stats flag. There are many more rsync options available, including inclusion or exclusion of patterns listed in an external file, throttling bandwidth, stopping after a large number of deletions, and so forth.

Probably the most confusing part of rsync is how it reacts to an ending slash character. The manual covers a bit of this confusion within the opening usage explanation. The following examples will hopefully make things a little clearer.

Either of these commands backs up the entire Library directory to ~/Backups/Library:

% rsync -vaz ~/Library ~/Backups
% rsync -vaz ~/Library ~/Backups/

Either of these commands backs up the contents of the Library directory into Backups (i.e., Backups/Application Support):

% rsync -vaz ~/Library/ ~/Backups
% rsync -vaz ~/Library/ ~/Backups/

There have been many tales of users who have lost a backup (or worse yet, horrifically screwed up a restoration with the --delete command), all due to a little slash being in the wrong place. When in doubt and experimenting with new rsync commands, use -n:

% rsync -vazn --delete ~/Library/ ~/Backups

The added -n says "hey, this is only a dry run, so show me what you're going to do, but don't actually do it." Be sure to do this with verbosity (-v) so that you can see exactly what would be deleted and added.