Re: Add jsonlog log_destination for JSON server logs

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Sehrope Sarkuni <sehrope(at)jackdb(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, david(at)fetter(dot)org
Subject: Re: Add jsonlog log_destination for JSON server logs
Date: 2021-09-13 02:22:37
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Sep 10, 2021 at 03:56:18PM +0900, Michael Paquier wrote:
> And this part leads me to the attached, where the addition of the JSON
> format would result in the addition of a couple of lines.

Okay, I have worked through the first half of the patch set, and
applied the improved versions of 0001 (refactoring of the chunk
protocol) and 0002 (addition of the tests for csvlog).

I have not looked in details at 0003 and 0004 yet. Still, here are
some comments after a quick scan.

+ * elog-internal.h
I'd rather avoid the hyphen, and use elog_internal.h.

+#define FORMATTED_TS_LEN 128
+extern char formatted_start_time[FORMATTED_TS_LEN];
+extern char formatted_log_time[FORMATTED_TS_LEN];
+void setup_formatted_log_time(void);
+void setup_formatted_start_time(void);
We could just use a static buffer in each one of those routines, and
return back a pointer to the caller.

+ else if ((Log_destination & LOG_DESTINATION_JSONLOG) &&
+ (jsonlogFile == NULL ||
+ time_based_rotation || (size_rotation_for & LOG_DESTINATION_JSONLOG)))
+ /* Write to JSON log if enabled */
+ else if (Log_destination & LOG_DESTINATION_JSONLOG)
+ {
Those bits in 0004 are wrong. They should be two "if" clauses. This
is leading to issues when setting log_destination to multiple values
with jsonlog in the set of values and logging_connector = on, and the
logs are not getting redirected properly to the .json file. We would
get the data for the .log and .csv files though. This issue also
happens with the original patch set applied on top of e757080. I
think that we should also be more careful with the way we free
StringInfoData in send_message_to_server_log(), or we are going to
finish with unwelcome leaks.

The same coding pattern is now repeated three times in logfile_rotate():
- Check if a logging type is enabled.
- Optionally open new file, with logfile_open().
- Business with ENFILE and EMFILE.
- pfree() and save of the static FILE* ane file name for each type.
I think that we have enough material for a wrapper routine that does
this work, where we pass down LOG_DESTINATION_* and pointers to the
static FILE* and the static last file names. That would have avoided
the bug I found above.

The rebased patch set, that has reworked the tests to be in line with
HEAD, also fails. They compile.

Sehrope, could you adjust that?

Attachment Content-Type Size
v3-0001-Split-csv-handling-in-elog.c-into-separate-csvlog.patch text/plain 21.4 KB
v3-0002-Add-jsonlog-log_destination-for-JSON-server-logs.patch text/plain 33.3 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2021-09-13 02:37:38 Re: Remove duplicate static function check_permissions in slotfuncs.c and logicalfuncs.c
Previous Message Amit Kapila 2021-09-13 02:15:51 Re: Added missing invalidations for all tables publication