teaser

Installing AWStats on Ubuntu 20.04 with Nginx

By Lars on Jun 17, 2020

Tags: Awstats, Nginx, Linux, Ubuntu, Droplet, Website hosting

This is a guide for installing AWStats on an Ubuntu 20.04 server for website analytics of multiple virtual hosts served by Nginx.

AWStats is a venerable log analyzer going all the way back to year 2000. But it is still one of the better free web analytics solutions that works without needing to insert any code on the websites to monitor. AWStats simply parses the web server access logs and is able to derive rather detailed information about unique page visits, referring URLs, search engine terms, visit duration, errors, download sizes and more.

To make AWStats tick, we need to accomplish the following things:

  1. Installation of AWStats and dependencies.
  2. Configuring AWStats generally and specifically for the websites to monitor
  3. Making AWStats parse the website logs at a certain interval
  4. Hosting the AWStats web interface behind a HTTP digest password

We'll go through all those steps in this guide.

For the record, the software versions used as of writing the guide are:

  • Ubuntu 20.04 LTS Server (in a DigitalOcean droplet)
  • Nginx 1.17.10
  • awstats 7.6 (build 20161204)
  • perl 5, version 30, subversion 0 (v5.30.0)
  • PHP 7.4.3

The guide relies heavily on information from these very useful posts:

1. Installing AWStats and dependencies

AWStats itself is available in the Ubuntu repositories. Install with:

  • sudo apt install awstats

We are going to turn on just one extra feature, the Geo IP locator. For this to work, the perl module Geo::IP is required. Here is how to install it using apt and cpan. First:

  • sudo apt install libgeoip-dev

When firing up cpan, note that AWStats is going to run as root so the modules must be installed in the scope of the root user, hence sudo:

  • sudo cpan

Let the installation proceed with default responses to the questions.

In the cpan terminal run:

  • install Geo::IP

Exit cpan and verify that Geo IP got installed as expected by checking the existence of: /usr/share/GeoIP/GeoIP.dat

Log format

Some guides recommend changing the access logging format for various purposes such as retrieving the correct IP for websites behind load balancers. I don't have such a setup, so I decided to stick with the default Nginx log format. Please see other guide for details about modifying the logging format.

Thus, in the Nginx server block for the websites to be monitored, the access_log setting should simply look like this:

access_log /var/log/nginx/example.com.access.log;

2. Configuring AWStats

AWStats has a huge number of configuration values. Most of them are general and just a few of them must be set specifically for each website. In this guide we will put the general settings and the website specific settings in separate files.

When AWStats is first installed there are two files in /etc/awstats:

  • awstats.conf - All configuration values
  • awstats.conf.local - An empty file supposed to be used for local customizations

We are going to do things slightly different. First, put everything into the .local file:

  • cd /etc/awstats
  • sudo rm awstats.conf.local
  • sudo mv awstats.conf awstats.conf.local

Now let's customize the awstats.conf.local file. I only changed the following values and set them like this:

  • LogFormat=1 This will work with the Nginx default logging format

  • DNSLookup=0 To turn off DNS lookups

  • LoadPlugin="geoip GEOIP_STANDARD /usr/share/GeoIP/GeoIP.dat" Find this line and uncomment it to enable Geo IP.

  • # Include "/etc/awstats/awstats.conf.local" Find this line, near the end of the file, and comment it. Otherwise there will be a config file include loop.

Individual website config files

For each website, a separate config file must be created with a filename exactly matching the website hostname, according to this format: /etc/awstats/awstats.${HOSTNAME}.conf.

For our "example.com" domain example, the configuration would look like this:

Contents of /etc/awstats/awstats.example.com.conf

Include "/etc/awstats/awstats.conf.local"

LogFile="/usr/share/awstats/tools/logresolvemerge.pl /var/log/nginx/example.com.access.log /var/log/nginx/example.com.access.log.1 |"

SiteDomain="example.com"

HostAliases="www.example.com"

The config file starts by including the generic settings file so the website specific settings can be overridden afterwards.

Note that LogFile is specified using a perl merging script that concatenates the ...access.log and ...access.log.1 log files. This approach is a brute force method to make sure AWStats gets to see all log entries even after log rotation.

Alternatively the log rotation in Nginx could be configured to call AWStats right before the rotation. I decided against this approach to keep things simple and separate, and I would like to update AWStats more often than the log rotation anyways.

Time for testing

Now we can try to run AWStats with the provided configuration:

  • sudo /usr/lib/cgi-bin/awstats.pl -config=example.com -update

It should succeed, and generate a "database" file in: /var/lib/awstats/

If it complains about missing access log files try:

  • sudo touch /var/log/nginx/example.com.access.log
  • sudo touch /var/log/nginx/example.com.access.log.1

There is a perl script to run AWStats on all the configured web sites, try it:

  • sudo /usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

3. Crontabbing AWStats

With a typical webserver configuration that rotates the access log every midnight, the minimum interval to run AWStats would be once a day. To get more frequent updates to the statistics, I decided to let it run it four times a day, at 00:30, 6:30, 12:30 and 18:30.

Here is how to do that using the root users crontab:

  • sudo crontab -u root -e

Type in the following:

# m h  dom mon dow   command
30 0,6,12,18 * * * /usr/share/doc/awstats/examples/awstats_updateall.pl now -awstatsprog=/usr/lib/cgi-bin/awstats.pl

4. Hosting AWStats's web interface

The goal here is to host the AWStats web interface using Nginx, through a cgi-bin PHP script for dynamically generated pages based on the "database" files in /var/lib/awstats/.

Generating a http login digest

Let's setup the http login digest first:

  • cd /etc/awstats
  • sudo su

Running the following one-liner will ask for username and password and append the login to awstats.htpasswd:

  • printf "`read -p Username:\ ; echo $REPLY`:`openssl passwd -apr1`\n" >> awstats.htpasswd

PHP cgi-bin handler script

Install PHP:

  • sudo apt install php-fpm

And create a file with the following contents named: /etc/nginx/cgi-bin.php 

<?php
$descriptorspec = array(
   0 => array("pipe", "r"),  // stdin is a pipe that the child will read from
   1 => array("pipe", "w"),  // stdout is a pipe that the child will write to
   2 => array("pipe", "w")   // stderr is a file to write to
);
$newenv = $_SERVER;
$newenv["SCRIPT_FILENAME"] = $_SERVER["X_SCRIPT_FILENAME"];
$newenv["SCRIPT_NAME"] = $_SERVER["X_SCRIPT_NAME"];
if (is_executable($_SERVER["X_SCRIPT_FILENAME"])) {
   $process = proc_open($_SERVER["X_SCRIPT_FILENAME"], $descriptorspec, $pipes, NULL, $newenv);
   if (is_resource($process)) {
       fclose($pipes[0]);
       $head = fgets($pipes[1]);
       while (strcmp($head, "\n")) {
           header($head);
           $head = fgets($pipes[1]);
       }
       fpassthru($pipes[1]);
       fclose($pipes[1]);
       fclose($pipes[2]);
       $return_value = proc_close($process);
   } else {
       header("Status: 500 Internal Server Error");
       echo("Internal Server Error");
   }
} else {
   header("Status: 404 Page Not Found");
   echo("Page Not Found");
}
?>

Nginx server block

Now we are ready to add a website server block in Nginx to host the awstats page. In this case the base URL will be awstats.example.com:

Example contents of /etc/nginx/sites-available/awstats

server {
    listen 80;
    listen [::]:80;

    server_name awstats.example.com;

    error_log /var/log/nginx/awstats.example.com.error.log;
    access_log off;
    log_not_found off;

    root /var/www/awstats.example.com;

    location ^~ /awstats-icon {
        alias /usr/share/awstats/icon/;
    }

    location ~ ^/([a-z0-9-_\.]+)$ {
        return 301 $scheme://awstats.example.com/cgi-bin/awstats.pl?config=$1;
    }

    location ~ ^/cgi-bin/.*\\.(cgi|pl|py|rb) {
        if ($args ~ "config=([a-z0-9-_\.]+)") {
            set $domain $1;
        }
        auth_basic            "Admin";
        auth_basic_user_file  /etc/awstats/awstats.htpasswd;

        location ~ ^/cgi-bin/.*\\.(cgi|pl|py|rb) {
            gzip off;
            include         fastcgi_params;
            fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
            fastcgi_index   cgi-bin.php;
            fastcgi_param   SCRIPT_FILENAME    /etc/nginx/cgi-bin.php;
            fastcgi_param   SCRIPT_NAME        /cgi-bin/cgi-bin.php;
            fastcgi_param   X_SCRIPT_FILENAME  /usr/lib$fastcgi_script_name;
            fastcgi_param   X_SCRIPT_NAME      $fastcgi_script_name;
            fastcgi_param   REMOTE_USER        $remote_user;
        }
    }
}

Enable the site:

  • sudo ln -s /etc/nginx/sites-available/awstats /etc/nginx/sites-enabled/

Test if Nginx is happy with the configuration:

  • sudo nginx -t

and restart Nginx if so:

  • sudo service nginx restart

Note that accessing "awstats.example.com" directly doesn't work and responds with 403 Forbidden.

The see statistics for a website, add the name after a slash:

  • awstats.example.com/example.com

Or for another monitored website on the same server:

  • awstats.example.com/anotherexample.com

HTTPS

Plain http digest logins like demonstrated above is not a very safe approach. Please enable https for the AWStats page, and all will be fine.

I hope you enjoyed this content!

ko-fi donate

Comments

Comments powered by Talkyard

  • ko-fi donate

Powered by Pelican         © 2024 by Lars Ole Pontoppidan