Re: Adding basic NUMA awareness

I withdrew my batched clock sweep on its own thread. The piece worth
keeping belongs here: replacing the 0..5 usage_count with a 1-bit
HOT/COOL cooling state, on top of your partitioning.

I built your v20260605 0001-0006 plus Wartak's 0008 and added the
evictor, then measured it against the same tree with the stock evictor.
Where it mattered I ran each test with and without partitioning to
separate an evictor effect from a NUMA effect. r8i.metal-96xl, 6 nodes,
your harness.

One result holds up. Under memory pressure HOT/COOL keeps more of the
frequently-used index pages resident than the 0..5 clock, and it does so
by the same margin with or without partitioning:

stock 0..5 HOT/COOL delta
unpartitioned:
index buffers resident 107,941 111,623 +3.4%
index share of evictions 36.89% 36.07% -0.83pt
6-NUMA partitioned:
index buffers resident 108,029 111,556 +3.3%
index share of evictions 36.88% 36.08% -0.80pt

Workload: 8GB cold heap, three 850MB secondary indexes, s_b=2GB, zipfian
index probes at 192 clients, 5 runs alternated. The per-run numbers
don't overlap (stock 36.87-36.90, HOT/COOL 36.06-36.14). Under 0..5 the
cold-heap fetches inflate usage counts and heap pages outlast index
pages by a sweep or two; the 1-bit state demotes them on the next pass,
so reused index pages win a little more often.

Because the margin is identical partitioned and not, it's a property of
the evictor, not of NUMA. So it's a small win on its own on
single-socket boxes, and it doesn't overlap with what partitioning buys
you. Throughput on top of partitioning is a wash on seqconcurrscans and
pgbenchS, and I'm not claiming otherwise.

One thing unrelated to the evictor that's useful to you: the c=1
seqconcurrscans regression gets worse with more nodes, -12% at 2 and
-49% at 6. Each partition is s_b/nodes, so at 6 nodes a single-client
scan no longer fits where it did unpartitioned. It looks like it wants a
floor on partition size rather than one partition per node. Happy to
test a floor if you want to try one.

Two build nits, both one-liners: pg_buffercache--1.7--1.8.sql updates
the Makefile but not meson.build, so CREATE EXTENSION fails under meson;
and 0008 needs a rebase (applies to master only with fuzz, leaves a
stray token in the ClockSweep struct).

Only 0009 is the change to the hot/cool algorithm, the other 8 patches
are what was on this list before.

best.

-greg