Re: query log corrupted-looking entries

From: "George Pavlov" <gpavlov(at)mynewplace(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-general(at)postgresql(dot)org>
Cc: "Ed L(dot)" <pgsql(at)bluepolka(dot)net>, "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>
Subject: Re: query log corrupted-looking entries
Date: 2007-06-08 16:30:21
Message-ID: 8C5B026B51B6854CBE88121DBF097A86DEA7AE@ehost010-33.exch010.intermedia.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us]
> What are the total lengths of the log entries in which you see the
> failure? (The "length" here includes all the lines belonging to a
> single logical entry, eg, ERROR, DETAIL, HINT.)

It is very hard to tease these apart because now that I look at it
closely it is a total mess; there are multiple interruptions and
interruptions inside of interruptions... The more I look at this the
worse it seems and now I am actually impressed that pgFouine is able to
not trip on most of these. It also makes me more suspect of the logs,
although laws of large numbers hopefully make them at least
directionally correct.

The interruption can happen anywhere, including the leading log
timestamp, so I am looking at the full log entry when counting. So,
counting from the beginning of the log entry, timestamp included, (e.g.
"2007-06-01 17:00:17 PDT [28955]: [1225-1] LOG: statement: PREPARE
<unnamed> AS SELECT ...") to the first interruption (the first start of
another log entry in the midst of the original one, e.g. up to and
including the "f" in "... left outer join f2007-06-01 17:00:19 PDT
[28920]: [1595-1] LOG: statement: EXECUTE <unnamed> [PREPARE: SELECT
PROPERTY_ID ...") I see all sorts of lengths, both smaller and larger
than 4096: I see interruptions at the 24th character, at the 399th,
4050th, 5063rd, etc. As to the full length of the entries that get
interrupted they do seem to be all on the long side--I can't say with
total certainty, but the couple of dozen that I looked at were all >
4096 when all the interruptions are taken out. So I think I can say that
I see corruptions happen within "long" entries, but they can happen
anywhere within that long entry, and one can have multiple interruptions
within one entry.

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Ed L. 2007-06-08 16:34:24 Re: query log corrupted-looking entries
Previous Message Andrew Sullivan 2007-06-08 15:14:52 Re: Disappearing table - suggestions on possible causes appreciated.

Browse pgsql-hackers by date

  From Date Subject
Next Message Ed L. 2007-06-08 16:34:24 Re: query log corrupted-looking entries
Previous Message Matthew T. O'Connor 2007-06-08 16:02:53 Re: COPYable logs status