Reducing apache log size

Posted Sat, 09/20/2008 - 18:13 by ma

Maintaining an Apache2 webserver takes a bit of fine tuning to get everything just how I like it. Apache has excellent logging capability and these logs, if properly maintained are an excellent information resource for any administrator.

However, if unmanaged these logs can get too large to handle. One of the things I like to do is use logrotate to set the time frames that my logs will rotate in, typically on a daily basis, and keep seven days worth of each log. I also use a bit of code in my httpd.conf file that calls the Apache module “SetEnvIf” to omit requests for certain file types from the access logs. This helps keep your logs small enough where you can work with them easily by ignoring requests for specified file types like .png, .jpg etc.

Add the section of code below to your httpd.conf and to each virtual host container on your server, edit it according to your needs.

<VirtualHost 1.2.3.4:80>
[...]
	SetEnvIf Request_URI \.xml dontlog
	SetEnvIf Request_URI \.jpg dontlog
	SetEnvIf Request_URI \.gif dontlog
	SetEnvIf Request_URI \.css dontlog
	SetEnvIf Request_URI \.png dontlog
	SetEnvIf Request_URI \.txt dontlog
	CustomLog /var/log/apache2/example.com-access.log combined env=!dontlog
	ErrorLog /var/log/apache2/example.com-error.log
[...]
</VirtualHost>

You can further trim down the logs with these : 

# Add the following code to httpd.conf to invoke the SetEnvIf module.
# Designate a remote address to ignore (content switch/load balancer?) if you choose.
# The line below ignores traffic with a source IP of: 192.168.252.1 (any x.x.x.x address will do)
SetEnvIf Remote_Addr “192\.168\.252\.1″ dontlog
# These lines ignore requests for .gif,.jpg, .png and .ico image files, add additional lines as needed.
# The “dontlog” variable is assigned to the file types listed.
SetEnvIf Request_URI \.gif dontlog
SetEnvIf Request_URI \.jpg dontlog
SetEnvIf Request_URI \.ico dontlog
SetEnvIf Request_URI \.png dontlog
SetEnvIf Request_URI \.css dontlog
# Log everything else except for what is marked by the “dontlog” variable to the “access_log”.
CustomLog logs/access_log combined env=!dontlog
# Ignore requests for the robots.txt file
SetEnvIf Request_URI “^/robots\.txt$” dontlog

DONT FORGET TO RESTART APACHE!!!