Re: Problems around compute_query_id

From: Julien Rouhaud <rjuju123(at)gmail(dot)com>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>
Subject: Re: Problems around compute_query_id
Date: 2021-04-12 06:56:59
Message-ID: 20210412065659.o4pnb7sknaxjzc42@nol
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Apr 12, 2021 at 03:12:40PM +0900, Michael Paquier wrote:
> Hi all,
> Starting a new thread as the one that has introduced compute_query_id
> is already long enough.
> Fujii-san has reported on Twitter that enabling the computation of
> query IDs does not work properly with log_statement as the query ID is
> calculated at parse analyze time and the query is logged before that.
> As far as I can see, that's really a problem as any queries logged are
> combined with a query ID of 0, and log parsers would not really be
> able to use that, even if the information provided by for example
> log_duration gives the computed query ID prevent in pg_stat_activity.

I don't see any way around that. The goal of log_statements is to log all
syntactically valid queries, including otherwise invalid queries. For
instance, it would log

SELECT notacolumn;

and there's no way to compute a queryid in that case. I think that
log_statements already causes some issues with log parsers. At least pgbadger
recommends to avoid using that:

"Do not enable log_statement as its log format will not be parsed by pgBadger."

I think we should simply document that %Q is not compatible with

> While making the feature run on some test server, I have noticed that
> %Q would log some garbage query ID for autovacuum workers that's kept
> around. That looks wrong.

I've not been able to reproduce it, do you have some hint on how to do it?

Maybe setting a zero queryid at the beginning of AutoVacWorkerMain() could fix
the problem?

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2021-04-12 06:58:28 Re: Truncate in synchronous logical replication failed
Previous Message Amit Kapila 2021-04-12 06:32:02 Re: missing documentation for streaming in-progress transactions