Re: Proposal: Adding json logging

On Mon, Apr 16, 2018 at 07:52:58PM -0400, Peter Eisentraut wrote:
> I have used https://github.com/mpihlak/pg_logforward in the past, which
> seems to be about the same thing.

Didn't know this one. Thanks.

> I have also had good success using syslog. While syslog is not very
> structured, the setting syslog_split_messages allows sending log entries
> that include newlines in one piece, which works well if you have some
> kind of full-text search engine at the receiving end.

syslog suffers from the possibility to lose messages if I recall
correctly, right? This may matter for some critical environments.

Upstream code has escape_json() directly included, which is able to do
the job and makes sure that a single JSON entry is not broken into
multiple lines. That's what my jsonlog uses to format the strings used,
and what I can see pg_logforward does as well witha custom copy.

As a whole model, producing one JSON object per line and per log-entry
is the most natural format in my opinion.

One thing which is perhaps sensitive for JSON is the timestamp format.
The JSON specification does not decide what should be the format of
timestamps, still parser facilities are somewhat all pointing into using
ISO 8601 with stuff like Javascript Date's toJSON method. There are
some side issues with the use of UTC.. So the thing is sorta of messy.

However, as JSON entries are usually larger than normal log entries,
getting log entries broken into multiple lines is easier if not using
logging_collector. People normally don't do that, but I received
complains on the matter as well when using Postgres in Docker container
for example. So documenting that logging_collector needs to be enabled
is important if this log format shows up in Postgres.
--
Michael