From: | Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org> |
---|---|
To: | John W Higgins <wishdev(at)gmail(dot)com> |
Cc: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
Subject: | Re: Proposal: Adding json logging |
Date: | 2018-04-18 19:54:05 |
Message-ID: | 20180418195405.2jnppct3brkfwyjr@alvherre.pgsql |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
John W Higgins wrote:
> On Sun, Apr 15, 2018 at 11:08 AM, David Arnold <dar(at)xoe(dot)solutions> wrote:
>
> > >This would appear to solve multiline issues within Fluent.....
> > >https://docs.fluentd.org/v0.12/articles/parser_multiline
> >
> > I definitely looked at that, but what guarantees do I have that the
> > sequence is always ERROR/STATEMENT/DETAIL? And not the other way round?
>
> Have you asked that question? You seem to at least have opened the source
> code - did you try to figure out what the logging format is?
I looked at this a couple of days ago. I think parsing with this
library is possible to a certain extent, and the problems stem from
limitations of the library. So, it turns out that the firstline can be
set to a single regex that searches for PANIC, FATAL, ERROR, WARNING,
LOG, NOTICE, DEBUG. That's always the first line in any postgres log
event.
A log event contains some subsequent lines. Those start either with a
tab (which is a continuation of the previous line) or with one of
DETAIL, CONTEXT, HINT, STATEMENT, QUERY. This seems very simple to
parse (just add lineN patterns for those), *except* that the messages
can be multiline too; and where would you assign the continuation lines
for each of those? parser_multiline does not support that.
Another thing worth keeping in mind is that you need to change the regex
depending on log_line_prefix, which sounds very painful.
All in all, the best approach might be to create a specific
parser_postgresql.rb plugin. Seems much easier to write two dozen lines
of Ruby than change all of PostgreSQL's logging infrastructure.
--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
From | Date | Subject | |
---|---|---|---|
Next Message | MauMau | 2018-04-18 19:59:34 | Re: Truncation failure in autovacuum results in data corruption (duplicate keys) |
Previous Message | Tom Lane | 2018-04-18 19:33:08 | Re: reloption to prevent VACUUM from truncating empty pages at the end of relation |