Re: Truncate logs by max_log_size

From: Jim Jones <jim(dot)jones(at)uni-muenster(dot)de>
To: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
Cc: Maxym Kharchenko <maxymkharchenko(at)gmail(dot)com>, Kirill Reshke <reshkekirill(at)gmail(dot)com>, Álvaro Herrera <alvherre(at)kurilemu(dot)de>, Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Kirill Gavrilov <diphantxm(at)gmail(dot)com>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>, Euler Taveira <euler(at)eulerto(dot)com>, pgsql-hackers(at)lists(dot)postgresql(dot)org
Subject: Re: Truncate logs by max_log_size
Date: 2026-07-09 07:41:57
Message-ID: e553fdd7-a26e-4a49-b6cf-0e1c8101381e@uni-muenster.de
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Thanks for the thorough review!

On 09/07/2026 06:03, Fujii Masao wrote:
> - errdetail("prepare: %s", pstmt->plansource->query_string);
> + char *truncated_stmt = truncate_query_log(pstmt->plansource->query_string);
> +
> + errdetail("prepare: %s", truncated_stmt ? truncated_stmt :
> pstmt->plansource->query_string);
>
> Shouldn't truncated_stmt be freed with pfree() (when non-NULL)
> immediately after errdetail()?

Yes, I missed that one.
Fixed.

> - is truncated to at most this many bytes.
> + is truncated to at most this many bytes. When a statement is
> + truncated, an ellipsis (<literal>...</literal>) is appended to
> + indicate that truncation has occurred.
>
> Since an ellipsis is appended, the logged output can exceed the configured
> limit by three bytes. Wouldn't it be clearer to say something like
> "has its statement text truncated to at most this many bytes"?
> That makes it clear the limit applies to the statement text, not the entire
> logged message.

Agreed.
I added your suggestion.

> -ok($node->log_contains(qr/statement: SELECT '123456789ABC$/m, $log_offset),
> +ok($node->log_contains(qr/statement: SELECT '123456789ABC...$/m, $log_offset),
>
> Shouldn't the dots be escaped? As written, "..." matches any three characters.
> I think it should be "\.\.\.".

At first I also thought so, as in this context ... could mean any three
arbitrary characters, but the tests results showed a different picture,
so I thought it was not necessary. Anyway, I escaped them now in v2.

> # a 24-byte query should end at the 20th byte ('C').
>
> Since an ellipsis is now appended, isn't it better to replace this
> with "a 24-byte query should be clipped at the 20th byte ('C')
> and followed by an ellipsis."?
>
>
> # With log_statement_max_length = 20, a 24-byte query should end
> # at the 20th byte ('C').
>
> Similarly, isn't it better to replace this with
> "With log_statement_max_length = 20, a 24-byte query should be clipped
> at the 20th byte ('C'). and followed by an ellipsis."?
>> * appending ellipsis to the query to indicate truncation (docs and tests
>> also updated accordingly). A side effect of this change is that setting
>> the parameter to 0 logs an ellipsis, which is not zero in length, but is
>> arguably correct, as it indicates that the whole query has been
>> truncated - just noting.
> So, should we update the docs and comments accordingly? For example:
>
> --------------------------------------------
> - A value of zero causes statements to be logged with an empty body.
> + A value of zero causes statements to be logged as only an ellipsis.
> <literal>-1</literal> (the default) logs statements in full.
> If this value is specified without units, it is taken as bytes.
>
> { name => 'log_statement_max_length', type => 'int', context =>
> 'PGC_SUSET', group => 'LOGGING_WHAT',
> - short_desc => 'Sets the maximum length in bytes of logged statements.',
> - long_desc => '-1 means log statement in full; 0 means log an empty
> statement body.',
> + short_desc => 'Sets the maximum length in bytes of logged statement text.',
> + long_desc => '-1 means log statement in full; 0 means log only an ellipsis.',
>
> - #log_statement_max_length = -1 # max bytes of logged statements;
> + #log_statement_max_length = -1 # max bytes of logged statement text;
> # -1 means log statement in full,
> - # 0 means log an empty body
> + # 0 means log only an ellipsis
>
> -# Verify 0 logs an empty statement body.
> +# Verify 0 logs only an ellipsis.
> note "Zero length truncation";
>
> ok($node->log_contains(qr/statement: \.\.\.\s*$/m, $log_offset),
> - "0 logs statement body with ellipsis");
> + "0 logs only an ellipsis");
> --------------------------------------------

Comments and docs updated.

>> BTW, should I open a new CF entry for this?
> Yes, I think that's a good idea so these follow-up patches don't get forgotten.

Done: https://commitfest.postgresql.org/patch/6998/

Best, Jim

Attachment Content-Type Size
v2-0001-Improve-log_statement_max_length-truncation.patch text/x-patch 9.1 KB
v2-0002-Simplify-callers-of-truncate_query_log.patch text/x-patch 3.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jelte Fennema-Nio 2026-07-09 07:47:14 Re: Don't use the deprecated and insecure PQcancel in our frontend tools anymore
Previous Message Peter Eisentraut 2026-07-09 07:20:29 Re: implement CAST(expr AS type FORMAT 'template')