Skip site navigation (1) Skip section navigation (2)

Re: Ragged latency log data in multi-threaded pgbench

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Ragged latency log data in multi-threaded pgbench
Date: 2010-03-18 18:49:28
Message-ID: 4BA275B8.5060000@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Takahiro Itagaki wrote:
> The log filenames are "pgbench_log.<main-process-id>.<thread-serial-number>"
> for each thread, but the first thread (including single-threaded) still uses
> "pgbench_log.<main-process-id>" for the name because of compatibility.
>   

Attached is an updated version that I think is ready to commit.  Only 
changes are docs--I rewrote those to improve the wording some.  The code 
looked and tested fine to me.  I just added support for the new format 
to pgbench-tools and am back to happily running large batches of tests 
using it again.

I confirmed a few things:

-On my CentOS system, the original problem is masked if you have 
"--enable-thread-safety" on; the multi-threaded output shows up without 
any broken lines into the single file.  As I suspected it's only the 
multi-process implementation that shows the issue here.  Since Tom 
points out that's luck rather than something that should be relied upon, 
I don't think that actually changes what to do here, it just explains 
why this wasn't obvious in earlier testing--normally I have thread 
safety on nowadays.

-Patch corrects the problem.  I took a build without thread safety on, 
demonstrated the issue with its pgbench.  Apply the patch, rebuild just 
pgbench, run again; new multiple log files have no issue.

-It's easy to convert existing scripts to utilize the new multiple log 
format.  Right now the current idiom you're forced into using when 
running pgbench scripts is to track the PID it's run as, then use 
something like:

mv pgbench_log.${PID} pgbench.log

To convert to a stable filename for later processing.  Now you just use 
something like this instead:

cat pgbench_log.${PID}* > pgbench.log
rm -f pgbench_log.${PID}*

And that works fine. 

-- 
Greg Smith  2ndQuadrant US  Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com   www.2ndQuadrant.us


Attachment: pgbench_log_20100318.patch
Description: text/x-patch (6.1 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Greg SmithDate: 2010-03-18 19:20:58
Subject: Re: Getting to beta1
Previous:From: Jeff DavisDate: 2010-03-18 18:21:07
Subject: Re: An idle thought

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group