| From: | ahmed <gouda0x(at)gmail(dot)com> |
|---|---|
| To: | Lukas Fittl <lukas(at)fittl(dot)com> |
| Cc: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Bernd Reiß <bd_reiss(at)gmx(dot)at> |
| Subject: | Re: Fix cumulative WAL statistics double-counting from parallel workers |
| Date: | 2026-10-31 01:44:25 |
| Message-ID: | CAFTkQVJgMxPVijJibj4odYBWLt9G0XAxKsMs4yGwJ-=8Zba02g@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi Lukas,
First, sorry for the previous incomplete email, sent by accident; please
ignore it.
> Turns out there is a current bug in master in how pg_stat_wal and
> pg_stat_get_backend_wal() report activity from parallel workers. This
> issue has existed since the introduction of pg_stat_wal in Postgres
> 14: We double count WAL produced in parallel workers, because we add
> WAL activity from the parallel workers to both the worker's pgWalUsage
> (direct) and the leader's pgWalUsage (via InstrAccumParallelQuery),
> and each of them emits that activity to the cumulative stats.
Nice Catch!
> Instead, at least for now, the easier solution seems to be to
> accumulate imported instrumentation data into both the existing
> pg(Wal|Buffer)Usage globals, and a separate set called
> pg(Wal|Buffer)UsageFromWorkers that counts only the activity from the
> workers. Any caller that wants to isolate the leader-only activity for
> reporting to the cumulative statistics system then subtracts the
> worker portion from the global instrumentation using a helper function
> (ComputeOwnWalUsage), and workers continue to report their own
> activity directly to the cumulative statistics system.
I was thinking: instead of introducing new pg(Wal|Buffer)UsageFromWorkers
global variables, could we make the existing `static WalUsage prevWalUsage`
a global variable, rename it to reportedWalUsage, and increment it directly
in InstrAccumParallelQuery() and on flush subtract it from pgWalUsage? That
would also minimize the number of subtraction operations on flush.
And we can do the same with pgStatBlock(Read|Write)Time, rename it to
reportedBlock(Read|Write)Time, etc…
Regards,
| From | Date | Subject | |
|---|---|---|---|
| Next Message | David Shorten | 2026-10-31 03:03:35 | Standby stuck after walreceiver restart just before a WAL segment boundary |
| Previous Message | ahmed | 2026-10-31 01:40:03 | Re: Fix cumulative WAL statistics double-counting from parallel workers |