Re: Fix cumulative WAL statistics double-counting from parallel workers

Hi,

On Thu, Oct 29, 2026 at 03:58:03PM -0700, Lukas Fittl wrote:
> Hi,
>
> Over in [0] we were recently discussing how parallel workers
> accumulate their buffer usage into the leader and how that makes it
> hard to pass pgBufferUsage instrumentation into the cumulative stats
> system, and I've also been looking into this for the stack-based
> instrumentation work [1], because it's really hard to avoid double
> counting due to how we accumulate into the leader's
> pgBufferUsage/pgWalUsage today.
>
> Turns out there is a current bug in master in how pg_stat_wal and
> pg_stat_get_backend_wal() report activity from parallel workers. This
> issue has existed since the introduction of pg_stat_wal in Postgres
> 14: We double count WAL produced in parallel workers, because we add
> WAL activity from the parallel workers to both the worker's pgWalUsage
> (direct) and the leader's pgWalUsage (via InstrAccumParallelQuery),
> and each of them emits that activity to the cumulative stats.

Ouch, nice catch!

> In terms of how to fix this, I first tried to invent a mechanism that
> would keep instrumentation imported from workers separate in the
> leader (so we never double count in the global). But that turns out to
> be pretty tricky, because we have use cases (pg_stat_statements being
> the main one, but e.g. also EXPLAIN ANALYZE of a query calling a
> function that itself runs a parallel query) where we want to track
> whole-statement activity (leader + workers), including utility
> statements, and we want to be able to do so in a nested manner.

Agreed.

> Instead, at least for now, the easier solution seems to be to
> accumulate imported instrumentation data into both the existing
> pg(Wal|Buffer)Usage globals, and a separate set called
> pg(Wal|Buffer)UsageFromWorkers that counts only the activity from the
> workers. Any caller that wants to isolate the leader-only activity for
> reporting to the cumulative statistics system then subtracts the
> worker portion from the global instrumentation using a helper function
> (ComputeOwnWalUsage), and workers continue to report their own
> activity directly to the cumulative statistics system.

Yeah that looks like the best way to proceed currently.

I wonder if the extra additions could be reduced though. What about first
accumulating all workers into a local BufferUsage/WalUsage and then calling
InstrAccumParallelQuery() once for all workers?

Also, worth to add some tests?

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com