Re: Fix cumulative WAL statistics double-counting from parallel workers

Hi Andres,

Thanks for taking a look!

On Wed, Nov 4, 2026 at 12:16 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> I'm not actually sure that its entirely the right thing to include the
> parallel worker WAL generated in this kind of example. If the parallel query
> directly triggers WAL it is counted, but if there's e.g. FDWs involved, it
> won't be. If DML triggers autovacuum doing the cleanup, it won't be, despite
> directly associated with DML happening inside a function. If checkpointer or
> bgwriter do a lot of IO triggered by a query, its not counted either. Those
> examples are all different, but it still feels all not quite right.

Yeah, that's a fair point - in the back of my mind I'm mostly thinking
of buffer usage reporting, where its much more likely users will see
the difference - WAL from parallel workers is more an edge case.

The architectural problem becomes a lot easier to solve if we don't
worry about putting WAL and buffer usage on the parent statement
EXPLAIN nodes, but e.g. instead displayed nested statements separately
with their activity under the main EXPLAIN plan.

I'll do some experimenting in that direction to see how much it'd buy
us in terms of reducing complexity - there is also Mohamed's patch on
that topic of how we display nested statements in EXPLAIN from a while
ago [0], but that was a rough draft (I think the actual implementation
needed a lot of work) and didn't really address this I think.

> > 1) pgWalUsage keeps today's meaning, pgWalUsageFromWorkers is
> > additionally accumulated in InstrAccumParallelQuery (the v1 patch)
> >
> > - When the function finishes executing, the worker instrumentation
> > gets fed into the parent process pgWalUsage and pgWalUsageFromWorkers
>
> I'm a bit confused about what you mean by that? Why would the end of the
> function need to do anything? I don't think your patch did anything like that?
> Did you just mean that it happend within the function, in one of the
> InstrAccumParallelQuery() calls?

Yeah, I was referencing the InstrAccumParallelQuery call that occurs
inside the function's statement that then causes the process
pgWalUsage to include the function's activity.

> I unfortunately don't immediately see a way other than 1) for the back
> branches.
>
> With your stack based query executor instrumentation infrastructure the big
> downside of 2) can afaict be addressed.

Ack, thanks for chiming in - I'll await for Michael to take another
look but will otherwise post an updated v2 addressing Bertrand's
comments, staying on the (1) approach.

> I think we really ought to make all access by extensions (be it our own or
> external ones) go through helper functions to get this kind of
> information. Right now they're way to tied to how we represent this
> information internally.

Agreed - I think we've actually made good progress on that in 19 with
Instrumentation now having a clear API that is usable outside of
executor instrumentation, so there is less need to read
pgWalUsage/pgBufferUsage directly.

The stack-based instrumentation series [1] actually has two prep
patches that drop the remaining direct users of pgBufferUsage to
further clean up the API (it keeps a user of pgWalUsage in
pruneheap.c, which is a special case), after which I think the main
unsolved use case is how we want the instrumentation fed into the
cumulative stats system without directly accessing a global variable.

Thanks,
Lukas

[0]: https://www.postgresql.org/message-id/flat/CAGnOmWp96ZHNyQ%2B4YijH9wqc%3DCcGm8cuBq9rRCXsaz0SO86tLg%40mail.gmail.com
[1]: https://www.postgresql.org/message-id/flat/CAP53Pkwd6Gd4OL6cgh449wtdznWitz_V%3D-PRoswLHOzsMbcftA%40mail.gmail.com#891ae8e91928207e840ef194bb90b29a

--
Lukas Fittl