Re: Redesign per-backend statistics

Hi,

On Tue, Sep 22, 2026 at 03:15:49AM +0000, Bertrand Drouvot wrote:
> Hi,
>
> On Tue, Sep 22, 2026 at 09:16:30AM +0900, Michael Paquier wrote:
> > On Mon, Sep 21, 2026 at 05:17:04AM -0400, Andres Freund wrote:
> > > On 2026-09-21 07:15:44 +0000, Bertrand Drouvot wrote:
> > >> +/* -------------------------------------------------------------------------
> > >> + *
> > >> + * pgstat_per_backend.c
> > >> + * Generic infrastructure for per-backend statistics.
> > >> + *
> > >> + * This file manages the dedicated per-kind dshashes used for per-backend
> > >> + * statistics, including entry creation, fetching, snapshots, transfer to
> > >> + * global statistics, and removal.
> > >
> > > It's not entirely obvious to me that a hash table is needed here. If I
> > > understand correctly, this just going to be used for per-backend versions of
> > > fixed stats. Which means we could just as well allocate all the memory in
> > > statically allocated shared memory at server start and have the per-backend
> > > stats be accessible by nothing more than an array access?
> >
> > Yeah. Array access based on an procnum index is still feeling like
> > the natural thing to do here (I think I've mentioned that upthread?).
>
> Yeah you did mention it. I took your reply in [1] as agreeing with the concern
> about reserving memory for mostly unused slots, though not necessarily with using
> a dshash. Is your preference still for the array despite that memory cost?
>
> [1]: https://postgr.es/m/anlVx_TD_Q8He5uf%40paquier.xyz

I had a chat with Andres about this thread at PGConf.EU last week, and I agree
that a fixed array is better than dshash. One reason is that dshash can also
consume a non negligible amount of memory.

For example, with 5000 maximum backends, the fixed array allocates about 16 MB.
Out of curiosity, I added some measurements to the dshash patch version and observed
that, with 3735 active client sessions, its memory usage jumps from 14 MB to 22 MB.
It was already about 10 MB with 1500 sessions.

So, PFA a new patch version making use of a fixed array and also adressing
Andres's comments upthread.

Regards,

--
Bertrand Drouvot
PostgreSQL Contributors Team
RDS Open Source Databases
Amazon Web Services: https://aws.amazon.com