Re: Proposal: Adding json logging

From: David Arnold <dar(at)xoe(dot)solutions>
To: Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Proposal: Adding json logging
Date: 2018-04-17 15:45:46
Message-ID: CAH6vsW+owB539ZkgUOo94u0cEJ__Fk_BaSE-0bHsNXZE_RRyWw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Alvaro, just to clarify for me, do you refer to the messages generated by
https://github.com/postgres/postgres/blob/master/src/backend/utils/error/elog.c
or
other messages?

Standardizing on UTF8 seems a good option. Assuming it* is* a problem, I
would classify this as another second-order problem, though, because it
would be also an issue right now, so we can sum up:

*Core-Problem:* "Multi line logs are unnecessarily inconvenient to parse
and are not compatible with the design of some (commonly used) logging
aggregation flows."
*2nd-order Problem 1:* "Logging space increasingly moves towards the
adoption of structured logging formats around json/logfmt. Compatibly
options (plural!) with main stream (not necessarily standard) tooling is a
value proposition of it's own kind. It helps increase odds of responsible
deployments and improves the overall experience in adopting PostgreSQL."
*2nd-order Problem 2:* "Encoding of logging can differ per database, this
inhibits the objective of reliable log stream parsing"

El mar., 17 abr. 2018 a las 9:26, Alvaro Herrera (<alvherre(at)alvh(dot)no-ip(dot)org>)
escribió:

> One issue I haven't seen mentioned in this thread is the translation
> status of the server message (as well as its encoding): it's possible to
> receive messages in some random language if the lc_message setting is
> changed. Requiring that lc_messages must always be set to some English
> locale seems like a poor answer to this problem.
> IMO the untranslated server message should be part of the event also.
> I don't know what to think of %-expansions of the message.
>
> The character encoding can be changed per database. Log files where the
> encoding differs across databases cannot be processed in any sane way.
> You can try some heuristics (try to read each message as utf8 first, and
> if that fails, then it must be Latin1! If that doesn't work for you,
> ... tough luck), but that's a pretty poor answer too. Not sure what is
> a good solution to this problem. Maybe ensure that these things are
> always UTF8?
>
> --
> Álvaro Herrera https://www.2ndQuadrant.com/
> PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
>
--
[image: XOE Solutions] <http://xoe.solutions/> DAVID ARNOLD
Gerente General
xoe.solutions
dar(at)xoe(dot)solutions
+57 (315) 304 13 68
*Confidentiality Note: * This email may contain confidential and/or private
information. If you received this email in error please delete and notify
sender.
*Environmental Consideration: * Please avoid printing this email on paper,
unless really necessary.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Fujii Masao 2018-04-17 15:46:58 Re: Speedup of relation deletes during recovery
Previous Message Alvaro Herrera 2018-04-17 15:30:53 Re: Test coverage for mark_invalid_subplans_as_finished