Reduce access_log noise

This is just a quick guide to eliminate certain activity from your access_log files.

Create a new file called filter.conf (where you put this depends on your apache deployment). Adapt the patterns as necessary. See apache docs for all the possible variables. You can pile up as many filters as you like.

SetEnvIFNoCase User-Agent "Googlebot" exclude_from_log
SetEnvIfNoCase Request_URI "\.(gif)|(jpg)|(png)|(css)|(ico)$" exclude_from_log
SetEnvIfNoCase Request_Method "(OPTIONS)|(PROPFIND)" exclude_from_log

This file should be included (probably at the global level) for easiest use. For OS provided apache, either put it in /etc/httpd/conf.d or explicitly include it into httpd.conf.

Anywhere you want the filter applied, simple alter you access_log definition to include the parameter after “combined”.

CustomLog "logs/access_log" combined env=!exclude_from_log

Restart apache to pick up the change and review your access_log to make sure it worked.

mod_rpaf

When Apache is behind a load balancer, frequently the load balancer IP will be recorded in the access/error logs instead of the actual client IP. This module modifies the inbound requests to show the actual client IP instead of the load balancer IP. This does not work for every load balancer.

This guide assumes you built Apache from source using my guide.

1. Download the mod_rpaf source from http://stderr.net/apache/rpaf/download/
2. Untar the source tarball
3. As root run

/usr/local/apache/bin/apxs -ci mod_rpaf-2.0.c

4. The module will be installed correctly in /usr/local/apache/modules
5. Create a new folder called custom under /usr/local/apache/conf
6. Create a file called httpd-modrpaf.conf in /usr/local/apache/conf/custom with the following content

LoadModule rpaf_module modules/mod_rpaf-2.0.so

RPAFenable On
RPAFsethostname On
RPAFproxy_ips 1.2.3.4
RPAFheader X-Forwarded-For

7. Replace 1.2.3.4 with the IP address of the load balancer.
8. Update the RPAFheader with the correct value if your load balancer does not pass “X-Forwarded-For”.
9. Make sure you include the new “custom” folder in your main httpd.conf

Include /usr/local/apache/conf/custom/*.conf

10. Restart Apache web server

mod_flvx

A custom module that enables FLV progressive streaming.

This guide assumes you built Apache from source using my guide.

1. Download the mod_flvx source from https://github.com/osantana/mod_flvx
2. As root run:

/usr/local/apache/bin/apxs -ci mod_flvx.c

3. The module will be installed correctly in /usr/local/apache/modules
4. Create a new folder called custom under /usr/local/apache/conf
5. Create a new file called httpd-modflvx.conf in /usr/local/apache/conf/custom with the following content

LoadModule flvx_module modules/mod_flvx.so
AddHandler flv-stream .flv

<IfModule mime_module%gt;
AddType video/x-flv .flv
</IfModule>

6. Make sure you include the new “custom” folder in your main httpd.conf

Include /usr/local/apache/conf/custom/*.conf

7. Restart apache web server

Apache post-build steps

If you followed my post on Apache from source, then you’ll also want to review these additional steps. Unless otherwise specified these steps are for modification of the httpd.conf file found in “/usr/local/apache/conf”.

1. Comment out the modules:

dav_module
dav_fs_module
proxy_ftp_module
proxy_scgi_module

2. Add the following excerpt to disable access to .svn/.git folder contents (put below similar rule for .ht* files):

#
# The following lines prevent .svn folders from being
# viewed by Web clients.
#
<DirectoryMatch "\.svn">
Order allow,deny
Deny from all
Satisfy All
</DirectoryMatch>

#
# The following lines prevent .git folders from being
# viewed by Web clients.
#
<DirectoryMatch "\.git">
Order allow,deny
Deny from all
Satisfy All
</DirectoryMatch>

5. Insert the following at the bottom of the file:

AddDefaultCharset Off

# Limit details, less details in response headers
ServerTokens prod

# Disable ETag exposure, this affects browser caching
Header unset ETag
FileETag None

# Disable these methods, this should be added to any vhosts defined
RewriteEngine on
RewriteCond %{REQUEST_METHOD} ^(TRACE|TRACK)
RewriteRule .* - [F]

Apache from source

These are my notes on building Apache (the web server, obviously) from source.

1. Install (or confirm installed):

openssl-devel
gcc

2. Download apache source from http://httpd.apache.org/download.cgi
3. Untar the source tarball
4. Navigate into extracted source
5. Run

./configure \
--prefix=/usr/local/apache \
--enable-so \
--enable-pie \
--enable-proxy \
--enable-ssl \
--enable-mods-shared=all \
--with-included-apr

If you want a worker version of Apache (instead of the prefork default), add the following to the command above (you’ll need a trailing slash on the previous line):

--with-mpm=worker

6. As root run:

make && make install

At this point you have a very stock Apache build. Most of the modules will be available as well. Look for additional articles on adding some useful modules.