Re: Fix cumulative WAL statistics double-counting from parallel workers

From: ahmed <gouda0x(at)gmail(dot)com>
To: Lukas Fittl <lukas(at)fittl(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Bernd Reiß <bd_reiss(at)gmx(dot)at>
Subject: Re: Fix cumulative WAL statistics double-counting from parallel workers
Date: 2026-10-31 01:44:25
Message-ID: CAFTkQVJgMxPVijJibj4odYBWLt9G0XAxKsMs4yGwJ-=8Zba02g@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Lukas,

First, sorry for the previous incomplete email, sent by accident; please
ignore it.

> Turns out there is a current bug in master in how pg_stat_wal and
> pg_stat_get_backend_wal() report activity from parallel workers. This
> issue has existed since the introduction of pg_stat_wal in Postgres
> 14: We double count WAL produced in parallel workers, because we add
> WAL activity from the parallel workers to both the worker's pgWalUsage
> (direct) and the leader's pgWalUsage (via InstrAccumParallelQuery),
> and each of them emits that activity to the cumulative stats.

Nice Catch!

> Instead, at least for now, the easier solution seems to be to
> accumulate imported instrumentation data into both the existing
> pg(Wal|Buffer)Usage globals, and a separate set called
> pg(Wal|Buffer)UsageFromWorkers that counts only the activity from the
> workers. Any caller that wants to isolate the leader-only activity for
> reporting to the cumulative statistics system then subtracts the
> worker portion from the global instrumentation using a helper function
> (ComputeOwnWalUsage), and workers continue to report their own
> activity directly to the cumulative statistics system.

I was thinking: instead of introducing new pg(Wal|Buffer)UsageFromWorkers
global variables, could we make the existing `static WalUsage prevWalUsage`
a global variable, rename it to reportedWalUsage, and increment it directly
in InstrAccumParallelQuery() and on flush subtract it from pgWalUsage? That
would also minimize the number of subtraction operations on flush.

And we can do the same with pgStatBlock(Read|Write)Time, rename it to
reportedBlock(Read|Write)Time, etc…

Regards,

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message David Shorten 2026-10-31 03:03:35 Standby stuck after walreceiver restart just before a WAL segment boundary
Previous Message ahmed 2026-10-31 01:40:03 Re: Fix cumulative WAL statistics double-counting from parallel workers