PATCH: pgbench - merging transaction logs

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: PATCH: pgbench - merging transaction logs
Date: 2015-03-08 20:29:15
Message-ID: 54FCB11B.5050907@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi there,

attached is a patch implementing merging of pgbench logs. These logs are
written by each thread, so with N threads you get N files with names

pgbench_log.PID
pgbench_log.PID.1
...
pgbench_log.PID.N

Before analyzing these logs, these files need to be combined. I usually
ended up wrinting ad-hoc scripts doing that, lost them, written them
again and so on over and over again.

The other disadvantage of the external scripts is that you have to pass
all the info about the logs (whether the logs are aggregated, whther
there's throttling, etc.).

So integrating this into pgbench directly seems like a better approach,
and the attached patch implements that.

With '-m' or '--merge-logs' on the command-line, the logs are merged at
the end, using a simple 2-way merge to keep the log sorted by the time
field. It should work with all the other options that influence the log
format (--rate, --aggregate-interval and --latency-limit).

I'll add this to CF 2016-06.

--
Tomas Vondra http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
pgbench-merge-logs-v1.patch text/x-diff 12.1 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tomas Vondra 2015-03-08 22:01:33 PATCH: pgbench - logging aggregated info and transactions at the same time
Previous Message Josh Berkus 2015-03-08 19:48:44 Re: MD5 authentication needs help -SCRAM