Re: Proposal: Adding json logging

This discussion is thriving, and long and behold, we've got an opinion from
Eduardo (fluent-bit):

https://github.com/fluent/fluent-bit/issues/564#issuecomment-381844419

>Also consider that in not all scenarios full multiline logs are flushed
right away, sometimes there are delays.

I think this line supports the view for multi line being a sub optimal
logging strategy (assessed globally and embedded into an deployment
ecosystem).

> Btw, we would be happy to work together with PostreSQL community to
support an official parser for your multiline logs

Although, fluent-bit seem to strive to work with postgres log "as it
pleases", I think it is still a valid problem definition against all
contrary arguments to maintain:

*Core-Problem:* "Multi line logs are unnecessarily inconvenient to parse
and are not compatible with the design of some (commonly used) logging
aggregation flows."
*2nd-order Problem:* "Logging space increasingly moves towards the adoption
of structured logging formats around json/logfmt. Compatibly options
(plural!) with main stream (not necessarily standard) tooling is a value
proposition of it's own kind. It helps increase odds of responsible
deployments and improves the overall experience in adopting PostgreSQL."

Please share you thoughts, if you still feel there are material objections
to the core problem? JSON or not JSON, as Christophe recalled, then is a
question in the solution space.
Note that part of the problem definition is "unnecessary", which implies
judgment on responsibilities and ecosystems working together, rather than a
broken system.

El mar., 17 abr. 2018 a las 7:31, David Arnold (<dar(at)xoe(dot)solutions>)
escribió:

> >To me it's implied by the doc at:
>
> https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
>
> Additionally this still depends on the way some middleware might choose to
> stream data. Can we really be sure the risk is minimal that any middleware
> would have chosen to treat new line as an entity delimitator?
>
> Can we even be sure that NO existing middleware would treat newline as a
> entity delimitator?
>
> I'm not that confident about that.
>
> Anticipating the possible argument, that the "others are wrong": This
> arguement, though valid, seems sometimes is tought it's limits in very
> mondane practicability and efficiency needs.
>
>
> El mar., 17 abr. 2018, 6:55 a.m., Daniel Verite <daniel(at)manitou-mail(dot)org>
> escribió:
>
>> David Arnold wrote:
>>
>> > Interesting, does that implicitly mean the whole log event would get
>> > transmitted as a "line" (with CRLF) in CSV.
>>
>> To me it's implied by the doc at:
>>
>> https://www.postgresql.org/docs/current/static/runtime-config-logging.html#RUNTIME-CONFIG-LOGGING-CSVLOG
>>
>> > In the affirmative scenario, this then would work for a true streaming
>> > aggregator (if CSV was supported).
>>
>> Assuming a real CSV parser tailing the log, there shouldn't be any trouble
>> with that.
>>
>>
>> Best regards,
>> --
>> Daniel Vérité
>> PostgreSQL-powered mailer: http://www.manitou-mail.org
>> Twitter: @DanielVerite
>>
> --
[image: XOE Solutions] <http://xoe.solutions/> DAVID ARNOLD
Gerente General
xoe.solutions
dar(at)xoe(dot)solutions
+57 (315) 304 13 68
*Confidentiality Note: * This email may contain confidential and/or private
information. If you received this email in error please delete and notify
sender.
*Environmental Consideration: * Please avoid printing this email on paper,
unless really necessary.