pgBadger 5: Analyze your logs daily with the incremental mode

Posted on 2014-02-10 by Dalibo

Paris, France - Feb. 10, 2014

DALIBO is proud to announce the release of pgBadger 5, a PostgreSQL performance analyzer, built for speed with fully detailed reports based on your Postgres log files.

This major version comes with a bunch of new metrics such as the SQL queries times histogram, some fixes in the new HTML5 design and the ability to build cumulative reports.

New incremental mode

The incremental mode is an old request issued at PgCon Ottawa 2012 that concern the ability to construct incremental reports with successive runs of pgBadger. It is now possible to run pgbadger once a day (or even every hours) and have cumulative reports per day and per week. A top index page allow you to go directly to the weekly and daily reports.

Here's screenshot of the new index page http://dalibo.github.io/pgbadger/screenshots/pgbadgerv5_index.png

This mode have been build with simplicity in mind. You just need to running pgbadger with cron as follow:

0 23 * * * pgbadger -q -I -O /var/www/pgbadger/ /var/log/postgresql.log

This is enough to have daily and weekly reports viewable using your browser.

Take a look at our demo here: http://dalibo.github.io/pgbadger/demov5/

There's also a useful improvement to allow pgBadger to seek directly to the last position in the same log file after a successive execution. This feature is only available using the incremental mode or the -l option and parsing a single log file. Let's say you have a weekly rotated log file and want to run pgBadger each days. With 2GB of log per day, pgbadger was spending 5 minutes per block of 2 GB to reach the last position in the log, so at the end of the week this feature will save you 35 minutes. Now pgBadger will start parsing new log entries immediately. This feature is compatible with the multiprocess mode using -j option (n processes for one log file).

New Histograms

This new major release adds some new metrics like an hourly graphic representation of the average count and duration of top normalized queries. Same for errors or events, you will be able to see graphically at which hours they are occurring the most often.

For example: http://dalibo.github.io/pgbadger/screenshots/pgbadgerv5_histogram.png

There's also a new "Histogram of query times", which is a new graph in the top queries slide that shows the query times distribution during the analyzed period.

For example: http://dalibo.github.io/pgbadger/screenshots/pgbadgerv5_histogram_2.png

There is also some graphic and report improvements, such as the mouse tracker formatting that have been reviewed. It now shows a vertical crosshair and all dataset values at a time when mouse pointer moves over series. Automatic queries formatting has also been changed, it is now done on double click event as simple click was painful when you wanted to copy some part of the queries.

Autovacuum reports now associate database name to the autovacuum and autoanalyze entries. Statistics now refer to "dbname.schema.table", previous versions was only showing the pair "schema.table".

This release also adds "Session peak" information and a report about "Simultaneous sessions". Parameters log_connections and log_disconnections must be enabled in postgresql.conf for this.

Links & Credits

DALIBO would like to thank the developers who submitted patches and the users who reported bugs and feature requests, especially Martin Prochazka, Herve Werner, tmihail, Reeshna Ramakrishnan, Guillaume Smet, Alexander Korotkov and Casey Allen Shobe.

pgBadger is an open project. Any contribution to build a better tool is welcome. You just have to send your ideas, features requests or patches using the GitHub tools or directly on our mailing list.


About pgBadger:

pgBagder is a new generation log analyzer for PostgreSQL, created by Gilles Darold (also author of ora2pg, the powerful migration tool). pgBadger is a fast and easy tool to analyze your SQL traffic and create HTML5 reports with dynamics graphs. pgBadger is the perfect tool to understand the behavior of your PostgreSQL servers and identify which SQL queries need to be optimized.

Docs, Download & Demo at http://dalibo.github.io/pgbadger/


About DALIBO:

DALIBO is the leading PostgreSQL company in France, providing support, trainings and consulting to its customers since 2005. The company contributes to the PostgreSQL community in various ways, including : code, articles, translations, free conferences and workshops

Check out DALIBO's open source projects at http://dalibo.github.io