The solution to the problem of wrangling logfiles is probably already installed on your server: it’s logrotate. Chances are logrotate is already configured in your system crontab as a daily task, and chances are it is configured to obey any configuration files found in the /etc/logrotate.d directory. If you find that directory, you are probably good to go.

In /etc/logrotate.d you’ll find several short configuration files. Each configuration deals with 1 or 2 related logfiles that should be checked and possibly administered on a regular basis. Logrotate can do alot of things with a logfile: clear the target log, create a copy of it, compress the copy, keep N backup copies, and it can be configured to rotate the target logfile on a daily, weekly, or monthly basis.

For example, here is how my client handles a logfile generated by a log4j-enabled java rmi server:

 /appdir/log/java_rmi_log.txt {
    daily
    rotate 10
    copytruncate
    compress
    notifempty
    missingok
 }

This configuration means

On a daily basis rotate, compress, and retain up to 10 days worth of logs using the copytruncate method. But notifempty; if the log is empty don’t do anything. Finally, missingok, so no need to log an error anywhere if the target logfile isn’t found.

This is nigh on a domain specific language for logfile rotation, and like most DSLs I’ve come across it’s much easier to read than it is to remember when writing. Don’t worry, the man page for logrotate lists all the details.