Apache / Nginx: Visualize Web Server Access Log In Real Time
Logstalgia (also known as ApachePong) is a very powerful and handy app. It is a website access log visualization tool. It is an extremely useful tool to give you look at your web server traffic. I often use this kind of software to justify and convince my clients and/or boss that we need more servers to handle traffic. It streams Apache / Lighttpd / Nginx web-server access logs as a pong-like battle between the web server and a never ending torrent of requests.
From the official project site:
Logstalgia is a website traffic visualization that replays web-server access logs as a pong-like battle between the web server and an never ending torrent of requests. Requests appear as colored balls (the same color as the host) which travel across the screen to arrive at the requested location. Successful requests are hit by the paddle while unsuccessful ones (eg 404 – File Not Found) are missed and pass through. The paths of requests are summarized within the available space by identifying common path prefixes. Related paths are grouped together under headings. For instance, by default paths ending in png, gif or jpg are grouped under the heading Images. Paths that don’t match any of the specified groups are lumped together under a Miscellaneous section.
Logstalgia requirements
- Workstation/Laptop/Server with a video card supporting OpenGL.
- A a fairly busy web-server to see interesting results.
- Linux operating systems. I tested this app on both Fedora and Debian/Ubuntu Linux.
Binary package based installation
Open a terminal and type the following command:$ sudo apt-get install logstalgia
Sample outputs:
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following NEW packages will be installed:
logstalgia
0 upgraded, 1 newly installed, 0 to remove and 4 not upgraded.
Need to get 161 kB of archives.
After this operation, 1,102 kB of additional disk space will be used.
Get:1 http://mirrors.kernel.org/debian/ stable/main logstalgia amd64 1.0.0-1+b1 [161 kB]
Fetched 161 kB in 2s (73.9 kB/s)
Selecting previously deselected package logstalgia.
(Reading database ... 338532 files and directories currently installed.)
Unpacking logstalgia (from .../logstalgia_1.0.0-1+b1_amd64.deb) ...
Processing triggers for man-db ...
Setting up logstalgia (1.0.0-1+b1) ...
Source code installation
You need to install the following software to build logstalgia. Open a terminal and type the following command:$ sudo apt-get install libsdl1.2-dev libsdl-image1.2-dev libpcre3-dev libftgl-dev libpng12-dev libjpeg62-dev
Download and build the software
Type the following wget command to download the software:$ cd /tmp
$ wget https://logstalgia.googlecode.com/files/logstalgia-1.0.3.tar.gz
Sample outputs:
--2013-04-30 15:17:41-- https://logstalgia.googlecode.com/files/logstalgia-1.0.3.tar.gz
Resolving logstalgia.googlecode.com... 74.125.135.82, 2404:6800:4001:c01::52
Connecting to logstalgia.googlecode.com|74.125.135.82|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 841822 (822K) [application/x-gzip]
Saving to: `logstalgia-1.0.3.tar.gz'
100%[======================================>] 8,41,822 456K/s in 1.8s
2013-04-30 15:17:44 (456 KB/s) - `logstalgia-1.0.3.tar.gz' saved [841822/841822]
Untar the tar ball with the tar command, enter:$ tar xvf logstalgia-1.0.3.tar.gz
First, make sure you have gcc compiler and related tools are installed. Type the following command to build software:$ cd logstalgia
$ ./configure
To compile logstalgia, type the following command:$ make
Finally, install logstalgia, enter:$ sudo make install
Sample outputs:
make[1]: Entering directory `/tmp/logstalgia-1.0.3'
test -z "/usr/local/bin" || /bin/mkdir -p "/usr/local/bin"
/usr/bin/install -c logstalgia '/usr/local/bin'
test -z "/usr/local/share/logstalgia/fonts" || /bin/mkdir -p "/usr/local/share/logstalgia/fonts"
/usr/bin/install -c -m 644 data/fonts/README data/fonts/FreeMonoBold.ttf data/fonts/FreeSerif.ttf '/usr/local/share/logstalgia/fonts'
test -z "/usr/local/share/logstalgia" || /bin/mkdir -p "/usr/local/share/logstalgia"
/usr/bin/install -c -m 644 data/ball.tga data/example.log data/glow.tga '/usr/local/share/logstalgia'
make install-data-hook
make[2]: Entering directory `/tmp/logstalgia-1.0.3'
mkdir -p -m 755 //usr/local/share/man/man1
/bin/sed 's|SDLAPP_RESOURCE_DIR|/usr/local/share/logstalgia|g' data/logstalgia.1 | gzip -f9 > /usr/local/share/man/man1/logstalgia.1.gz
make[2]: Leaving directory `/tmp/logstalgia-1.0.3'
make[1]: Leaving directory `/tmp/logstalgia-1.0.3'
How do I use logstalgia?
The syntax is as follows:
logstalgia access_log
logstalgia /var/www/httpd/access_log
logstalgia [options] /var/www/httpd/access_log
Examples
To watch /var/www/apache2/access.log file using the default settings, enter:$ logstalgia /var/www/apache2/access.log
You can force logstalgia to read logs from STDIN:$ tail -f /var/log/httpd/access_log | logstalgia -
OR$ tail -f /var/log/httpd/access_log | logstalgia --sync
How do I watch the live access.log from the remote server?
Use the following syntax over the ssh session for piracy and security reasons:
ssh vivek@server1.cyberciti.biz tail -f /var/log/apache2/access.log | logstalgia --sync
ssh vivek@server1.cyberciti.biz tail -f /var/log/nginx/www.cyberciti.bizaccess.log | logstalgia --sync
How do I save logstalgia output as a video file?
You can create a video of Logstalgia using the --output-ppm-stream option. The syntax is to create ppm file:$ logstalgia -1280x720 --output-ppm-stream output.ppm /var/log/httpd/access.log
OR use the ssh command for remote web-server:$ ssh user@server1.cyberciti.biz tail -f /var/log/nginx/www.cyberciti.biz_access.log | logstalgia -1280x720 --output-ppm-stream --sync output.ppm
Finally, use the ffmpeg command as follows to create .mp4 file from .ppm file:$ ffmpeg -y -r 60 -f image2pipe -vcodec ppm -i output.ppm -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 nginx.server.log.mp4
To play the file nginx.server.log.mp4 use mplayer command, type:$ mplayer nginx.server.log.mp4
Keyboard shortcuts
Interactive keyboard commands are follows:
- c : Displays Logstalgia logo.
- n : Jump forward in time to next log entry.
- + or - : Adjust simulation speed.
- < or > : Adjust time scale.
- Esc : Quit the app.
Type the following command to see all other supported options:$ logstalgia --help
Check out related media
This tutorial is also available in a quick video format:
Video 01: Linux / Unix Web Server Log File Visualization in Realtime With Logstalgia
Other realtime logfile visualization software for *nix
- glTail.rb – a real-time logfile visualization written in Ruby.
- goaccess – a real-time open source web log analyzer and interactive viewer.
References
You should follow me on twitter here or grab rss feed to keep track of new changes.
Featured Articles:
Apache / Nginx: Visualize Web Server Access Log In Real Time
Apache / Nginx: Visualize Web Server Access Log In Real Time