Re: Logging parallel worker draught

From: Benoit Lobréau <benoit(dot)lobreau(at)dalibo(dot)com>
To: Melanie Plageman <melanieplageman(at)gmail(dot)com>, Sami Imseih <samimseih(at)gmail(dot)com>
Cc: Tomas Vondra <tomas(at)vondra(dot)me>, Guillaume Lelarge <guillaume(at)lelarge(dot)info>, Alena Rybakina <a(dot)rybakina(at)postgrespro(dot)ru>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, "Imseih (AWS), Sami" <simseih(at)amazon(dot)com>, Alvaro Herrera <alvherre(at)alvh(dot)no-ip(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "Andrey M(dot) Borodin" <x4mmm(at)yandex-team(dot)ru>
Subject: Re: Logging parallel worker draught
Date: 2025-04-14 14:41:31
Message-ID: e8b0f127-697a-4d18-a718-1c96425a343b@dalibo.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 4/7/25 6:41 PM, Melanie Plageman wrote:
> On Mon, Feb 3, 2025 at 12:37 AM Sami Imseih <samimseih(at)gmail(dot)com> wrote:
> I started looking at this, and I like the idea.

Thanks for taking a look!

> A few comments: I don't understand what 0002 is. For starters, the
> commit message says something about pg_stat_database, and there are no
> changes related to that.

I had originally split this part out while working on the patch to add
parallel worker stats in pg_stat_database [1], in order to isolate the
common components. In the end, that patch only accounted for user queries.

I merged it into "Implements logging for parallel worker usage in
utilities" for v9.

> Also, we already have basically identical logging coming from
> parallel_vacuum_process_all_indexes() and viewable in existing output.
> Not only does your implementation not replace this, it is odd that
> setting your new guc to none does not disable this. It seems a bit
> inconsistent. I'm not sure what the exact right behavior is here,
> though.

That logging is used for the VERBOSE mode of VACUUM. There was also
dicussion to add similar info for parallel index creation.

The use case here is different — the goal is to audit parallel worker
usage across the entire instance, without needing every call site to use
VACUUM (VERBOSE) along with SET log_min_messages = info.

I avoided reusing that part of the code because I thought the
expectation was to aggregate worker counts and display them in
parallel_vacuum_end(). Sami also mentionned that using the same log
line everywhere in the patch would make parsing easier, which I tought
was a good idea.

> Since your last update, it seems parallel gin index build has been
> committed, so perhaps you want to add that.

Thanks for the heads-up! I've added logging in _gin_end_parallel().

You’ll find the updated patch attached.

[1] https://commitfest.postgresql.org/patch/5212/

--
Benoit

Attachment Content-Type Size
v9-0001-Add-a-guc-for-parallel-worker-logging.patch text/x-patch 7.0 KB
v9-0002-Implements-logging-for-parallel-worker-usage-in-util.patch text/x-patch 4.0 KB
v9-0003-Implements-logging-for-parallel-worker-usage-in-quer.patch text/x-patch 1.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Daria Shanina 2025-04-14 14:55:17 Re: rounding_up
Previous Message Robert Haas 2025-04-14 14:38:29 Re: Performance issues with v18 SQL-language-function changes