Recipe 8.5 Trailing a Growing File

8.5.1 Problem

You want to read from a continually growing file, but the read fails when you reach the current end-of-file.

8.5.2 Solution

Read until end-of-file. Sleep, clear the EOF flag, and read some more. Repeat until interrupted. To clear the EOF flag, either use seek:

for (;;) {
    while (<FH>) { .... }
    sleep $SOMETIME;
    seek(FH, 0, 1);
}

or use the IO::Handle module's clearerr method:

use IO::Handle;

for (;;) {
    while (<FH>) { .... }
    sleep $SOMETIME;
    FH->clearerr( );
}

8.5.3 Discussion

When you read until end-of-file, an internal flag is set that prevents further reading. The most direct way to clear this flag is the clearerr method, if supported: it's in the IO::Handle modules.

$naptime = 1;

use IO::Handle;
open (LOGFILE, "/tmp/logfile") or die "can't open /tmp/logfile: $!";
for (;;) {
    while (<LOGFILE>) { print }     # or appropriate processing
    sleep $naptime;
    LOGFILE->clearerr( );            # clear stdio error flag
}

Because Perl v5.8 ships with its own stdio implementation, that simple approach should almost always work. On the rare system where it doesn't work, you may need to use seek. The seek code given in the Solution tries to move zero bytes from the current position, which nearly always works. It doesn't change the current position, but it should clear the end-of-file condition on the handle so that the next <LOGFILE> operation picks up new data.

If that still doesn't work, perhaps because it relies on features of your I/O implementation, you may need to use the following seek code, which remembers the old file position explicitly and returns there directly.

for (;;) {
    for ($curpos = tell(LOGFILE); <LOGFILE>; $curpos = tell(LOGFILE)) {
        # process $_ here
    }
    sleep $naptime;
    seek(LOGFILE, $curpos, 0);  # seek to where we had been
}

On some kinds of filesystems, the file could be removed while you are reading it. If so, there's probably little reason to continue checking whether it grows. To make the program exit in that case, stat the handle and make sure its link count (the third field in the return list) hasn't gone to 0:

exit if (stat(LOGFILE))[3] =  = 0

If you're using the File::stat module, you could write that more readably as:

use File::stat;
exit if stat(*LOGFILE)->nlink =  = 0;

The CPAN module File::Tail lets you tie a filehandle so that the read operation blocks at the end of the file until more data is available:

use File::Tail;

tie *FH, "File::Tail", (name => $FILENAME);
while (<FH>) {
  # do something with line read
}

The <FH> operator in this case never returns undef to indicate end-of-file.

8.5.4 See Also

The seek and tell functions in perlfunc(1) and in Chapter 29 of Programming Perl; your system's tail(1) and stdio(3) manpages; the documentation for the standard File::stat module (also in Chapter 32 of Programming Perl); the documentation for the CPAN module File::Tail