Skip site navigation (1) Skip section navigation (2)

Re: query log corrupted-looking entries

From: "Ed L(dot)" <pgsql(at)bluepolka(dot)net>
To: "George Pavlov" <gpavlov(at)mynewplace(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: query log corrupted-looking entries
Date: 2007-06-01 21:20:19
Message-ID: 200706011520.19227.pgsql@bluepolka.net (view raw or flat)
Thread:
Lists: pgsql-generalpgsql-hackers
On Friday 01 June 2007 3:09 pm, George Pavlov wrote:
> On 5/29/2007 10:19 AM, Ed L. wrote:
> > On Wednesday 23 May 2007 1:04 pm, George Pavlov wrote:
> > FWIW, I've also been seeing this sort of query log
> > corruption for as long as I can remember, 7.1 through 8.2,
> > HPUX (parisc, ia64), Linux on intel, amd...
>
> Do you have any tricks for dealing with the problem from a
> query analyzer perspective? That is, if you use something like
> pgfouine do you have any quick and easy way to remove those
> lines (and the affected lines around them)? Or someway to
> "fix" the corrupted lines? I'd say that lately ~40% of my
> daily query logs are suffering from this problem making query
> analysis very hard.

Not really.  I filter by perl regular expressions, but I wouldn't 
say that is reliable.  However, our query volumes are high 
enough that losing a few here and there is no big deal in 
analysis.  I long ago realized that reliably replaying query 
logs was infeasible due in part to this issue.  Regex filtering 
is problematic and error-prone (for example, how do you reliably 
tell the difference between log timestamp and DB data?).

Perhaps others have better answers.

Ed

In response to

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2007-06-01 21:31:52
Subject: Upcoming events
Previous:From: Andrew DunstanDate: 2007-06-01 21:15:26
Subject: syslogger line-end processing infelicity

pgsql-general by date

Next:From: Alexander StauboDate: 2007-06-01 21:22:18
Subject: Re: Slightly OT.
Previous:From: Tom LaneDate: 2007-06-01 21:15:04
Subject: Re: Seq Scan

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group