At a customer site, the test and production linux servers for some intranet applications were slowly running out of disk space. The apps themselves were running fine, it was the logfiles generated by the apps that were the issue; logging that came in useful just in case something went wrong, but quickly aged. In one particular case, a catalina.out file was several years old and was 11 gigabytes; 90% of it wasn’t really relevant any longer.
The solution to the problem of wrangling logfiles is probably already installed on your server: it’s logrotate. Chances are logrotate is already configured in your system crontab as a daily task, and chances are it is configured to obey any configuration files found in the /etc/logrotate.d directory. If you find that directory, you are probably good to go.
In /etc/logrotate.d you’ll find several short configuration files. Each configuration deals with 1 or 2 related logfiles that should be checked and possibly administered on a regular basis. Logrotate can do alot of things with a logfile: clear the target log, create a copy of it, compress the copy, keep N backup copies, and it can be configured to rotate the target logfile on a daily, weekly, or monthly basis.
For example, here is how my client handles a logfile generated by a log4j-enabled java rmi server:
/appdir/log/java_rmi_log.txt {
daily
rotate 10
copytruncate
compress
notifempty
missingok
}
This configuration means
On a daily basis rotate, compress, and retain up to 10 days worth of logs using the copytruncate method. But notifempty; if the log is empty don’t do anything. Finally, missingok, so no need to log an error anywhere if the target logfile isn’t found.
This is nigh on a domain specific language for logfile rotation, and like most DSLs I’ve come across it’s much easier to read than it is to remember when writing. Don’t worry, the man page for logrotate lists all the details.