From: | Gilles Darold <gilles(at)darold(dot)net> |
---|---|
To: | David Fetter <david(at)fetter(dot)org>, Gilles Darold <gilles(at)darold(dot)net> |
Cc: | PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: [PATCH] Log CSV by default |
Date: | 2018-12-03 20:20:01 |
Message-ID: | 17cccd7d-c66d-cd30-6b67-c34a72c14143@darold.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
Le 03/12/2018 à 19:25, David Fetter a écrit :
> On Mon, Dec 03, 2018 at 07:18:31PM +0100, Gilles Darold wrote:
>> Le 30/11/2018 à 19:53, David Fetter a écrit :
>>> This makes it much simpler for computers to use the logs while not
>>> making it excessively difficult for humans to use them.
>> I'm also not very comfortable with this change. For a pgBadger point
>> of view I don't know an existing library to parse csv log in
>> multi-process mode. The only way I know is to split the file or load
>> chunks in memory which is not really recommended with huge log
>> files. I am not saying that this is not possible to have a Perl CSV
>> library allowing multi-process but this require some additional days
>> of work.
> How about other structured log formats? Would one for JSON work
> better?
pgBadger is able to parse jsonlog
(https://github.com/michaelpq/pg_plugins/tree/master/jsonlog) output in
multi-process mode because it do not use an external library to parse
the json. In this minimalist implementation I assume that each log line
is a full json object. Honestly I do not have enough feedback on this
format to be sure that my basic jsonlog parser is enough, maybe it is
not. What is really annoying for a log parser is the preservation of
newline character in queries, if the format outputs each atomic log
messages on a single line each, this is ideal.
>
>> Also at this time I have not received any feature request for that,
>> maybe the use of csvlog format is not so common.
> Part of the reason it's uncommon is that it's not the default we ship
> with, and defaults influence this very much.
I agree. I also think that syslog or stderr are more human readable than
csvlog or jsonlog, which have probably contributed to their large
adoption too.
--
Gilles Darold
http://www.darold.net/
From | Date | Subject | |
---|---|---|---|
Next Message | Stephen Frost | 2018-12-03 21:20:47 | Re: [PATCH v19] GSSAPI encryption support |
Previous Message | Pavel Stehule | 2018-12-03 19:16:37 | citelny zajimavy clanek venovany fyzicke replikaci |