Re: WIP - syslogger infrastructure changes

On Fri, Sep 25, 2009 at 4:29 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Fri, Sep 25, 2009 at 22:26, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>> On Fri, Sep 25, 2009 at 4:13 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>>> On Fri, Sep 25, 2009 at 05:18, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>> On Mon, Sep 14, 2009 at 4:14 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>>>>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>>>>> First, the patch removes the logging_collector parameter and basically
>>>>>> assumes that logging_collector is always on.
>>>>>
>>>>> I don't find that to be a good idea, and you certainly have not made
>>>>> a case why we should change it.  I can't see any reason why pushing
>>>>> functionality out of backends and downstream to the syslogger process
>>>>> is an improvement.  What it's more likely to do is create a processing
>>>>> bottleneck and a single point of failure.
>>>>
>>>> Hmm.  I think the justification was supposed to be this part here:
>>>>
>>>> $ With that, it's no longer necessary to restart your server just to
>>>> $ reconfigure the logging, and it also takes away a confusing parameter
>>>> $ (really "log_destination=stderr, logging_collector=on" is *not* a logical
>>>> $ way to say "log to file"). Instead, it adds a log_destination of "file" that
>>>> $ is the standard log to file."
>>>>
>>>> Do we have any positive or negative experience with logging_collector
>>>> as a performance bottleneck?  Are there people running with
>>>> logging_collector=off to avert disaster?
>>>
>>> I've never heard of that, so I'd be very interested in hearing if somebody did.
>>>
>>> Actually, it's not starting the logging collector that's the issue.
>>> It's the redirection.
>>>
>>> So we could always start the logging collector, but not necessarily
>>> redirect stderr. For those who then set logging to "file", it gets
>>> passed to the logging collector. Those who set it to "stderr" just
>>> have to deal with stderr somehow (pg_ctl -l for example). It's still
>>> going to require a restart to change stderr, but that will only hit
>>> those people who are actually having performance issues from it  (and
>>> want to switch to using stderr instead of logging collector).
>>>
>>> Then we just send the data to the syslogger over a separate pipe. If
>>> we keep the "endpoint" of this pipe in the postmaster, a new syslogger
>>> could just "pick that up" after a crash, no? Data that goes on stderr
>>> will still go to stderr and not be captured in this case, but normally
>>> that would be no data at all, and if it did happen it'd be caught by
>>> wherever you point pg_ctl -l at (which is the same place the early
>>> startup messages goes, before the logger is running).
>>
>> That doesn't sound horribly robust.  I thought part of the point of
>> the design of the logging collector was that if random libraries
>> linked into the backend emit messages on stderr, we'll still capture
>> them and send them on to the appropriate place, even though we don't
>> really know what they are.  Having such messages go off to some
>> probably-much-less-carefully-monitored logfile somewhere doesn't seem
>> good.
>
> Well, from how I read the first complaint here, running with
> logging_collector on is simply broken from the crash perspective, no?
> I mean, don't we have the same problem *today*, it's just that
> logging_collector isn't on by default (in the source distribution that
> is - on our most installed platform it is..)

AIUI the problem is that when logging_collector is on, we throw away
the original stderr. That's OK as long as you never try to switch
back to it.

> Also, how often does it actually happen that "random libraries linked
> into the backend emit messages on stderr"? Is this really something
> that we need to pay so much attention to, rather than just let them go
> on stderr and have them be redirected?

I think so. But I'm willing to be told I'm full of it.

...Robert