Re: shared-memory based stats collector - v69

On Tue, Apr 5, 2022 at 4:16 PM Andres Freund <andres(at)anarazel(dot)de> wrote:

> On 2022-04-05 14:43:49 -0700, David G. Johnston wrote:
> > On Tue, Apr 5, 2022 at 2:23 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> >
> > > I guess I should add a paragraph about snapshots / fetch consistency.
> > >
> >
> > I apparently confused/combined the two concepts just now so that would
> help.
>
> Will add.
>
>
Thank you.

On a slightly different track, I took the time to write-up a "Purpose"
section for pgstat.c :

It may possibly be duplicating some things written elsewhere as I didn't go
looking for similar prior art yet, I just wanted to get thoughts down.
This is the kind of preliminary framing I've been constructing in my own
mind as I try to absorb this patch. I haven't formed an opinion whether
the actual user-facing documentation should cover some or all of this
instead of the preamble to pgstat.c (which could just point to the docs for
prerequisite reading).

David J.

* Purpose:

* The PgStat namespace defines an API that facilitates concurrent access
* to a shared memory region where cumulative statistical data is saved.
* At shutdown, one of the running system workers will initiate the writing
* of the data to file. Then, during startup (following a clean shutdown)
the
* Postmaster process will early on ensure that the file is loaded into
memory.
*
* Each cumulative statistic producing system must construct a PgStat_Kind
* datum in this file. The details are described elsewhere, but of
* particular importance is that each kind is classified as having either a
* fixed number of objects that it tracks, or a variable number.
*
* During normal operations, the different consumers of the API will have
their
* accessed managed by the API, the protocol used is determined based upon
whether
* the statistical kind is fixed-numbered or variable-numbered.
* Readers of variable-numbered statistics will have the option to locally
* cache the data, while writers may have their updates locally queued
* and applied in a batch. Thus favoring speed over freshness.
* The fixed-numbered statistics are faster to process and thus forgo
* these mechanisms in favor of a light-weight lock.
*
* Cumulative in this context means that processes must, for numeric data,
send
* a delta (or change) value via the API which will then be added to the
* stored value in memory. The system does not track individual changes,
only
* their net effect. Additionally, both due to unclean shutdown or user
request,
* statistics can be reset - meaning that their stored numeric values are
returned
* to zero, and any non-numeric data that may be tracked (say a timestamp)
is cleared.