Setting up AWStats on Tomcat

Blog
5/6/06
Lluis Turró Cutiller
82.688
7
java statistics

Earlier 2005 I had the need to install web statistics for Tomcat web servers. The reason might be obvious: XMLPortal runs on JSP/Servlet containers. Usually, Java hosting plans use Apache server to generate access logs and then redirect requests to Tomcat. This runs pretty well, but once a hosting provider offers to improve your account by sending requests directly to Tomcat, you can't say no. And I said yes. Soon I start missing things, like web statistics. This is how, step by step, I dealt with it. Today I have splendid AWStats on my web site!

Requirements

  • Perl 5.2 or above
  • Tomcat 5 (I can't see any reason for Tomcat 4 not to work, I haven't tried it, though)
  • awstats.war

Changing Tomcat access logs format

Tomcat access logs format isn't exactly what we need. We could also have modified AWStats configuration files, but with the loss of important information. Thus, let's make Tomcat logs compatible with AWStats. Logs are configured in [Tomcat_install]/conf/server.xml. They might look something like:


<Valve className="org.apache.catalina.valves.AccessLogValve"
            directory="logs"  prefix="localhost_access_log." suffix=".txt"
            pattern="common" resolveHosts="false"/>
Normally this lines are commented. Just take in mind they should appear, uncommented, within your Host definition.

We will replace the valve definition with this one:


<Valve className="org.apache.catalina.valves.AccessLogValve" 
            directory="logs" prefix="www_mydomain_com_access_log." suffix=".log" 
            pattern="combined" fileDateFormat="dd-MM-yy" resolveHosts="false"/>

Enable Perl scripts

Tomcat has disabled by default CGI executions. This is because CGIs do not conform Tomcat security box. But we know exactly what we are going to do, aren't we ?

The only thing you need to do is renaming [Tomcat_install]/server/lib/servlets-cgi.renametojar to [Tomcat_install]/server/lib/servlets-cgi.jar. This will enable AWStats Perl scripts to execute on Tomcat server. When it's done, restart Tomcat.

There is something more done through awstats.war file's web.xml. You might want to look at this file once downloaded.

Install AWStats

AWStats is included in awstats.war file, therefore you will not need to install AWStats by hand.

Deploy awstats.war file. Will suffice copying this file into [Tomca_install]/webapps folder, but in case your Tomcat is configured not to auto-deploy war files, unzip it yourself. By default would create an [Tomca_install]/webapps/awstats folder.

Although you could change default context, isn't a good idea. At least not to start with. Better make it work and then think about changing things.

Configure AWStats

This is the hardest part, did you already take a coffee?

First we will make some assumptions, are these ones:

  • awstats.war was deployed into /awstats default context. See Install AWStats.
  • Our domain is www.mydomain.com.

Now, go to [Tomca_install]/webapps/awstats/WEB-INF/cgi-bin folder. Next instructions make the assumption this is your current working folder.

  1. Copy awstats.model.conf into awstats.www.mydomain.com.conf.
  2. Edit awstats.www.mydomain.com.conf.

You can use any text editor of your preference. Now, we will change some file default values. You must look for the attributes and change their values as shown below:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/*.log |"
LogType=W
LogFormat=1
LogSeparator=" "
SiteDomain="www.mydomain.com"
DNSLookup=1
DirData="."
DirCgi="/cgi-bin"
DirIcons="/awstats/icon"
Remember replacing [TI] with full path to Tomcat. And www.mydomain.com with your domain name.

Update site statistics.

Here you have two choices:

  1. Set AllowToUpdateStatsFromBrowser=1 in awstats.www.mydomain.com.conf file, see above. This will allow updating from the web browser with: http://www.mydomain.com/awstats/cgi-bin/awstats.pl?config=www.mydomain.com&update=1.
  1. Using Telnet or SSH, go to [Tomca_install]/webapps/awstats/WEB-INF/cgi-bin folder and run perl awstats.pl -config=www.mydomain.com -update.

View site statistics.

Type http://www.mydomain.com/awstats/cgi-bin/awstats.pl?config=www.mydomain.com. That was all.

Creating separated domain statistics.

From the instructions above, everything is valid for creating separated domain statistics, except for one thing:

Inside your awstats.www.mydomain.com.conf we said:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/*.log |"

This line was gathering the whole set of log files, and this is against having statistics apart for each domain. We should replace this line to match the log files for the corresponding domain, as we said when changing Tomcat access logs format:

LogFile="[TI]/webapps/awstats/WEB-INF/tools/logresolvemerge.pl [TI]/logs/www_mydomain_com*.log |"

Important! Don't omit the asterisk in pointing the log files. Take in mind that Tomcat will create an access log file for each day, and that final file's name will include the date.

Troubleshooting

This tip was contributed by Jeff Cann

When I ran awstats for the first time, from command line, within tomcat container a 'Permission denied' error occurred. Here's screen output:

root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/cgi-bin-> perl awstats.pl
-config=www.ultimatescorecard.com -update
Update for config "./awstats.www.ultimatescorecard.com.conf"
With data in log file
"/var/tomcat5/webapps/awstats/WEB-INF/tools/logresolvemerge.pl
/var/tomcat5/logs/access_log.txt |"...
Error: Couldn't open server log file
"/var/tomcat5/webapps/awstats/WEB-INF/tools/logresolvemerge.pl
/var/tomcat5/logs/access_log.txt |" : Permission denied
Setup ('./awstats.www.ultimatescorecard.com.conf' file, web server or
permissions) may be wrong.
Check config file, permissions and AWStats documentation (in 'docs'
directory).

It occurred to me that the logresolvemerge.pl was not set executable:

[root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/tools-> ls -l
-rw-r--r--    1 nobody   nobody      18471 Jun  5 23:11
awstats_buildstaticpages.pl
-rw-r--r--    1 nobody   nobody      25865 Jun  5 23:11 awstats_configure.pl
-rw-r--r--    1 nobody   nobody      12768 Jun  5 23:11 awstats_exportlib.pl
-rw-r--r--    1 nobody   nobody       5199 Jun  5 23:11 awstats_updateall.pl
-rw-r--r--    1 nobody   nobody      27518 Jun  5 23:11 logresolvemerge.pl
-rw-r--r--    1 nobody   nobody      27175 Jun  5 23:11 maillogconvert.pl
-rw-r--r--    1 nobody   nobody      10205 Jun  5 23:11 urlaliasbuilder.pl
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 webmin
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 xslt

So, to fix the problem, I did a chmod 755 *.pl in the awstats/WEB-INF/tools/ directory.

[root@jumanji]/var/tomcat5/webapps/awstats/WEB-INF/tools-> ll
total 148
-rwxr-xr-x    1 nobody   nobody      18471 Jun  5 23:11
awstats_buildstaticpages.pl
-rwxr-xr-x    1 nobody   nobody      25865 Jun  5 23:11 awstats_configure.pl
-rwxr-xr-x    1 nobody   nobody      12768 Jun  5 23:11 awstats_exportlib.pl
-rwxr-xr-x    1 nobody   nobody       5199 Jun  5 23:11 awstats_updateall.pl
-rwxr-xr-x    1 nobody   nobody      27518 Jun  5 23:11 logresolvemerge.pl
-rwxr-xr-x    1 nobody   nobody      27175 Jun  5 23:11 maillogconvert.pl
-rwxr-xr-x    1 nobody   nobody      10205 Jun  5 23:11 urlaliasbuilder.pl
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 webmin
drwxr-xr-x    2 nobody   nobody       4096 Jun  5 23:11 xslt

Comentaris

mamurillo@actionsdata.com 1/3/07
Step by step process works with Tomcat 4.1 but it show me and error with web updates. Something like pipe error with pl file who process tomcat logs. Manual update works perfectly.I suspect I will have to do a cron job... Thanks!
glenaldo 2/5/07
Hi Lluis, The link for the awstats.war file appears to be broken.. Could you possibly check this? I am not running apache and a war based deploy is exactly what I need tia. G.
shiuhrong@synamatix.com 20/5/07
The requested resource (/awstats/cgi-bin/awstats.pl) is not available. i got this error when i run it...
ctjay@pookmail.com 11/12/07
I just wanted to thank you for this great tutorial. I followed your directions and was able to install and run this perfectly. You did an awesome job and really helped me with every detail. Thank you again!
Hendra effendi 31/3/08
Great tutorial for starters. Good job. The official Awstat documentation does not give a good explanation for tomcat installation
mrmanishs@gmail.com 11/8/08
I used this tutorial, which is great, to install awstats about a year ago. However, I noticed that the latest version of awstats (6.8) has some stats that I would like to incorporate. The WAR file here is 6.2. Are there any hints on how I can upgrade the installation from 6.2 to 6.8? Manish
PreecrElD 6/2/11
hi, new to the site, thanks.