From: | Andrew Dunstan <andrew(at)dunslane(dot)net> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Alvaro Herrera <alvherre(at)commandprompt(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: COPYable logs status |
Date: | 2007-06-11 14:59:48 |
Message-ID: | 466D6364.60106@dunslane.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers pgsql-patches |
Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
>
>> The idea of one pipe per process is not really workable, because it
>> would mean having as many pipes as backends which does not sound very
>> good. But how about a mixed approach -- like have the all the backends
>> share a pipe, controlled by an LWLock, and the auxiliary process have a
>> separate pipe each?
>>
>
> Multiple pipes seem like a mess, and in any case the above still doesn't
> work for stderr output produced by non-cooperative software (dynamic
> loader for instance).
>
> The only solution that I can see is to invent some sort of simple
> protocol for the syslogger pipe. Assume that the kernel honors PIPE_BUF
> (this assumption may need proving, see other message). We could imagine
> having elog.c divvy up its writes to the pipe into chunks of less than
> PIPE_BUF bytes, where each chunk carries info sufficient to let it be
> reassembled. Perhaps something on the order of
>
> \0 \0 2-byte-length source-PID end-flag text...
>
> The syslogger reassembles these by joining messages with the same
> origination PID, until it gets one with the end-flag set. It would need
> enough code to track multiple in-progress messages.
>
> The logger would have to also be able to deal with random text coming
> down the pipe (due to aforesaid non-cooperative software). I would be
> inclined to say just take any text not preceded by \0\0 as a standalone
> message, up to the next \0\0. Long chunks of non-protocol text would
> risk getting treated as multiple messages, but there's probably not a
> lot of harm in that.
>
> BTW, exactly what is the COPYable-logs code going to do with random
> text? I trust the answer is not "throw it away".
>
>
>
The CSVlog pipe is a separate pipe from the stderr pipe. Anything that
goes to stderr now will continue to go to stderr, wherever that is.
I like this scheme for a couple of reasons:
. it will include the ability to tell the real end of a message
. it will let us handle non-protocol messages (although there shouldn't
be any in the CSVlog pipe).
I'll try to get a patch out for just the stderr case, which should be
back-patchable, then adjust the CSVlog patch to use it.
I'm thinking of handling the partial lines with a small dynahash of
StringInfo buffers, which get discarded whenever we don't have a partial
line for the PID.
cheers
andrew
From | Date | Subject | |
---|---|---|---|
Next Message | Andrew Dunstan | 2007-06-11 15:01:42 | Re: What's with the StartDb:2 failures on skylark? |
Previous Message | Magnus Hagander | 2007-06-11 14:41:42 | Re: What's with the StartDb:2 failures on skylark? |
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2007-06-11 15:11:59 | Re: WIP: updatable cursors in plpgsql |
Previous Message | Pavel Stehule | 2007-06-11 14:52:15 | Re: WIP: updatable cursors in plpgsql |