Re: shared memory based stat collector (was: Sharing record typmods between backends)

Andres Freund <andres(at)anarazel(dot)de> writes:
> You both are obviously right. Another point of potential concern could
> be that we'd pretyt much fully rely on dsm/dht's being available, for
> the server to function correctly. Are we ok with that? Right now
> postgres still works perfectly well, leaving parallelism aside, with
> dynamic_shared_memory_type = none.

In principle we could keep the existing mechanism as a fallback.
Whether that's worth the trouble is debatable. The current code
in initdb believes that every platform has some type of DSM support
(see choose_dsm_implementation). Nobody's complained about that,
and it certainly works on every buildfarm animal. So for all we know,
dynamic_shared_memory_type = none is broken already.

> I think there's two other relevant points here:

> a) It'd be quite useful to avoid needing a whole cluster's stats in
> memory. Even if $subject would save memory, I'm hesitant committing
> to something requiring all stats to be in memory forever. As a first
> step it seems reasonable to e.g. not require state for all databases
> to be in memory. The first time per-database stats are required, it
> could be "paged in". Could even be more aggressive and do that on a
> per-table level and just have smaller placeholder entries for
> non-accessed tables, but that seems more work.

Huh? As long as we don't lock the shared memory into RAM, which on most
platforms we couldn't do without being root anyway, ISTM the kernel should
do just fine at paging out little-used areas of the stats. Let's not
reinvent that wheel until there's demonstrable proof of need.

> b) I think our tendency to dump all stats whenever we crash isn't really
> tenable, given how autovacuum etc are tied to them.

Eh, maybe. I don't think crashes are really so common on production
systems. As developers we have an inflated view of their frequency ;-)

regards, tom lane