Setting up AWStats on Tomcat

Earlier 2005 I had the need to install web statistics for Tomcat web servers. The reason might be obvious: XMLPortal runs on JSP/Servlet containers. Usually, Java hosting plans use Apache server to generate access logs and then redirect requests to Tomcat. This runs pretty well, but once a hosting provider offers to improve your account by sending requests directly to Tomcat, you can't say no. And I said yes. Soon I start missing things, like web statistics. This is how, step by step, I dealt with it. Today I have splendid AWStats on my web site!

Requirements

• Perl 5.2 or above
• Tomcat 5 (I can't see any reason for Tomcat 4 not to work, I haven't tried it, though)
• awstats.war

Changing Tomcat access logs format

Tomcat access logs format isn't exactly what we need. We could also have modified AWStats configuration files, but with the loss of important information. Thus, let's make Tomcat logs compatible with AWStats. Logs are configured in [Tomcat_install]/conf/server.xml. They might look something like:

<Valve className="org.apache.catalina.valves.AccessLogValve"
            directory="logs"  prefix="localhost_access_log." suffix=".txt"
            pattern="common" resolveHosts="false"/>

We will replace the valve definition with this one:

<Valve className="org.apache.catalina.valves.AccessLogValve" 
            directory="logs" prefix="www_mydomain_com_access_log." suffix=".log" 
            pattern="combined" fileDateFormat="dd-MM-yy" resolveHosts="false"/>

Enable Perl scripts

Tomcat has disabled by default CGI executions. This is because CGIs do not conform Tomcat security box. But we know exactly what we are going to do, aren't we ?

The only thing you need to do is renaming [Tomcat_install]/server/lib/servlets-cgi.renametojar to [Tomcat_install]/server/lib/servlets-cgi.jar. This will enable AWStats Perl scripts to execute on Tomcat server. When it's done, restart Tomcat.

Install AWStats

AWStats is included in awstats.war file, therefore you will not need to install AWStats by hand.

Deploy awstats.war file. Will suffice copying this file into [Tomca_install]/webapps folder, but in case your Tomcat is configured not to auto-deploy war files, unzip it yourself. By default would create an [Tomca_install]/webapps/awstats folder.

Configure AWStats

This is the hardest part, did you already take a coffee?

First we will make some assumptions, are these ones:

• awstats.war was deployed into /awstats default context. See Install AWStats.
• Our domain is www.mydomain.com.

Now, go to [Tomca_install]/webapps/awstats/WEB-INF/cgi-bin folder. Next instructions make the assumption this is your current working folder.

1. Copy awstats.model.conf into awstats.www.mydomain.com.conf.
2. Edit awstats.www.mydomain.com.conf.

You can use any text editor of your preference. Now, we will change some file default values. You must look for the attributes and change their values as shown below:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/*.log |"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="www.mydomain.com"
DNSLookup=1
DirData="."
DirCgi="/cgi-bin"
DirIcons="/awstats/icon"

Update site statistics.

Here you have two choices:

1. Set AllowToUpdateStatsFromBrowser=1 in awstats.www.mydomain.com.conf file, see above. This will allow updating from the web browser with: http://www.mydomain.com/awstats/cgi-bin/awstats.pl?config=www.mydomain.com&update=1.

1. Using Telnet or SSH, go to [Tomca_install]/webapps/awstats/WEB-INF/cgi-bin folder and run perl awstats.pl -config=www.mydomain.com -update.

View site statistics.

Type http://www.mydomain.com/awstats/cgi-bin/awstats.pl?config=www.mydomain.com. That was all.

Creating separated domain statistics.

From the instructions above, everything is valid for creating separated domain statistics, except for one thing:

Inside your awstats.www.mydomain.com.conf we said:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/*.log |"

This line was gathering the whole set of log files, and this is against having statistics apart for each domain. We should replace this line to match the log files for the corresponding domain, as we said when changing Tomcat access logs format:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/www_mydomain_com*.log |"

Important! Don't omit the asterisk in pointing the log files. Take in mind that Tomcat will create an access log file for each day, and that final file's name will include the date.

Troubleshooting

When I ran awstats for the first time, from command line, within tomcat container a 'Permission denied' error occurred. Here's screen output:

root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/cgi-bin-> perl awstats.pl
-config=www.ultimatescorecard.com -update
Update for config "./awstats.www.ultimatescorecard.com.conf"
With data in log file
"/var/tomcat5/webapps/awstats/WEB-INF/tools/logresolvemerge.pl
/var/tomcat5/logs/access_log.txt |"...
Error: Couldn't open server log file
"/var/tomcat5/webapps/awstats/WEB-INF/tools/logresolvemerge.pl
/var/tomcat5/logs/access_log.txt |" : Permission denied
Setup ('./awstats.www.ultimatescorecard.com.conf' file, web server or
permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs'
directory).

It occurred to me that the logresolvemerge.pl was not set executable:

[root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/tools-> ls -l
-rw-r--r--    1 nobody   nobody      18471 Jun  5 23:11
awstats_buildstaticpages.pl
-rw-r--r--    1 nobody   nobody      25865 Jun  5 23:11 awstats_configure.pl
-rw-r--r--    1 nobody   nobody      12768 Jun  5 23:11 awstats_exportlib.pl
-rw-r--r--    1 nobody   nobody       5199 Jun  5 23:11 awstats_updateall.pl
-rw-r--r--    1 nobody   nobody      27518 Jun  5 23:11 logresolvemerge.pl
-rw-r--r--    1 nobody   nobody      27175 Jun  5 23:11 maillogconvert.pl
-rw-r--r--    1 nobody   nobody      10205 Jun  5 23:11 urlaliasbuilder.pl
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 webmin
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 xslt

So, to fix the problem, I did a chmod 755 *.pl in the awstats/WEB-INF/tools/ directory.

[root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/tools-> ll
total 148
-rwxr-xr-x    1 nobody   nobody      18471 Jun  5 23:11
awstats_buildstaticpages.pl
-rwxr-xr-x    1 nobody   nobody      25865 Jun  5 23:11 awstats_configure.pl
-rwxr-xr-x    1 nobody   nobody      12768 Jun  5 23:11 awstats_exportlib.pl
-rwxr-xr-x    1 nobody   nobody       5199 Jun  5 23:11 awstats_updateall.pl
-rwxr-xr-x    1 nobody   nobody      27518 Jun  5 23:11 logresolvemerge.pl
-rwxr-xr-x    1 nobody   nobody      27175 Jun  5 23:11 maillogconvert.pl
-rwxr-xr-x    1 nobody   nobody      10205 Jun  5 23:11 urlaliasbuilder.pl
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 webmin
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 xslt