Re: [HACKERS] WIP: Separate log file for extension

From: Antonin Houska <ah(at)cybertec(dot)at>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] WIP: Separate log file for extension
Date: 2017-11-24 14:01:16
Message-ID: 31753.1511532076@localhost
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Antonin Houska <ah(at)cybertec(dot)at> wrote:

> Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> > > On Fri, Aug 25, 2017 at 12:12 AM, Antonin Houska <ah(at)cybertec(dot)at> wrote:
> >
> > I like this idea in general.
> >
> > Then it's supposed to change some of its attributes
> >
> > > adjust_log_stream_attr(&stream->filename, "my_extension.log");
> >
> > This, however, seems to be wrong.
> >
> > The logfile name does not belong in the extension, it belongs in the
> > configuration file. I think the extension should set it's "stream id" or
> > whatever you want to call it, and then it should be possible to control in
> > postgresql.conf where that log is sent.
> Doesn't the last paragraph of
> address your concerns?

Besides a new version of the patch, an example extension is attached that uses
the feature.

> > Also, what if this extension is loaded on demand in a session and not via
> > shared_preload_libraries? It looks like the syslogger only gets the list of
> > configured streams when it starts?
> Yes, the syslogger gets the list of streams only when it starts, so the
> extension that wants to use this feature needs to provide the file information
> via shared_preload_libraries. I consider it sufficient because various
> existing logging-related GUCs also can't be changed on-the-fly.
> > In short, I think the solution should be more generic, and not "just for extensions".

statement_log.diff demonstrates how the feature can be used by various
subsystems of PG core. Please consider it an example rather than part of the
"separate log patch". Even if there were no other design questions, there's
too much copy & paste in guc.c. I have no good idea right now how to improve
this part.

> o.k. Any idea about dividing the streams into categories? Should they for
> example correspond somehow to categories of GUC variables?
> > I completely missed this thread when I did my quick-wip at
> >
> > -- some of the changes made were close enough that I got the two confused :)
> > Based on the feedback of that one, have you done any performance checks?
> I don't expect mere routing of messages into multiple files to bring any
> overhead. I'll run some tests, just out of curiosity.

After having read the thread on your patch I think that the reason you were
asked to evaluate performance was that your patch can possibly make syslogger
a bottleneck. In contrast, my patch does not prevent user from disabling the
syslogger if it (the syslogger) seems to cause performance issues.

Antonin Houska
Cybertec Schönig & Schönig GmbH
Gröhrmühlgasse 26
A-2700 Wiener Neustadt

Attachment Content-Type Size
separate_log.diff text/x-diff 82.2 KB
log_gen.tgz application/x-gzip 2.2 KB
statement_log.diff text/x-diff 5.1 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Oliver Ford 2017-11-24 14:11:17 Add RANGE with values and exclusions clauses to the Window Functions
Previous Message John Naylor 2017-11-24 13:40:12 Re: scan-build plpython stuff