Re: [PATCH] Batched clock sweep to reduce cross-socket atomic contention

Hi,

On 2026-07-29 13:56:41 -0400, Greg Burd wrote:
> AI NOTICE:
> Yes, this patch was co-developed/refined/tested/benchmarked using
> AI/LLMs and even this email started off as LLM content. All the work
> is steered by me and reviewed before submission by me, I stand behind
> it warts and all. (gulp) ;-P

Not a fan. I'm ok with some closely supervised AI written code, but AI written
emails I tend to just put to the bottom of my pile (which I rarely reach). I
can't tell how much vetting goes into them, and I have too much to do to be an
AI sanity checker.

> To find out whether the *mechanism* is real at all, I built a
> sweep-bound microbenchmark: a function that pins+releases random blocks
> of a table 1.7x shared_buffers with the normal (non-ring) strategy, so
> StrategyGetBuffer is essentially the only work. 360 concurrent loops (=
> cores on the instance), 64GB shared_buffers, 6 NUMA nodes:
>
> build block-evictions/sec vs stock
> stock 2,096,314 --
> bcs batch=1 2,138,485 +2% (single fetch-add: parity)
> bcs batch=16 3,531,021 +68%
> bcs batch=64 3,664,458 +75%

Seems pretty doubtful this is measuring something super interesting, given
that you'd a) use a lot of memory bandwidth for that much IO b) there's a lot
of other contention points for that much IO (e.g. buffer table partition
locks).

A 75% win actually seems somewhat disappointing for such a microbenchmark? I'd
bet that you'd see WAY bigger gains with properly partitioning this (since the
batching still leaves you with a lot of c2c accesses).

I still don't see what the point of pursuing batching instead of helping out
with Tomas' partitioned sweep patch is.

>
> 0002 -- cooling evictor: why the 0..5 counter, and the failure mode
> ========================================================================
>
> Ants, you said the initial results were a promising indication that
> G-CLOCK's 0..5 counter isn't carrying much value, and asked for a wider
> access-pattern gamut. Here's the strongest thing I found.

Bla.

> The 0..5 counter has a pathological failure mode that the 1-bit HOT/COOL
> clock does not. Stock must decrement a buffer from 5 to 0 before it can
> evict it -- up to five visits per victim -- and it resets its
> bounded-scan counter on every decrement, so under a hot,
> continuously-evicting pool it never gives up early. With a large
> shared_buffers and huge_pages off (a multi-GB BufferDescriptors array on
> 4KB pages), those repeated visits are dTLB misses. I measured
> ticks-per-victim climbing to ~9 on stock under load where the 1-bit
> clock stays flat (it demotes HOT->COOL in one pass; a victim is produced
> in at most ~3 ticks, and typically 1-2). In the extreme this turns into
> a throughput cliff / multi-second p99 for stock -- the field reports of
> "raise shared_buffers and TPS falls off" on large no-huge-pages
> machines. The 1-bit clock has no such cliff because it does bounded
> work per victim by construction.

That's a lot of words with no actual evidence what the real world cost of that
is.

> On scan resistance, which is the reason the 0..5 counter's replacement
> has to be careful: a demand-loaded page is admitted COOL (probationary)
> and only promoted to HOT on a genuine second touch, so a one-touch
> sequential scan fills and drains the COOL stage without displacing the
> HOT working set. I tested this adversarially -- an OLTP hot set that
> fits in shared_buffers, hit by point lookups, while a concurrent seqscan
> streams a table 4x shared_buffers, on local NVMe (no EBS/IOPS cap
> masking the eviction rate):
>
> OLTP hit ratio under the scan: stock 99.999% bcs 99.998%
> OLTP resident set: flat in both

I don't think it makes much sense to measure this when you can achieve close
to 100 hit rate, since that's so easy to achieve. There's dozens of ways to
achieve scan resistance that work well in that scenario but that fall entirely
completely flat when you have actual cache pressure.

I think this algorithm will *trivially* fail in a lot of cases without
strategies. Once you have sufficient cache pressure the likelihood for
cold->hot promotion ends up being too low to happen reliably before cold pages
are evicted. And because this doesn't have something like a ghost directory,
it'll not even have a chance to detect and fix that the next time the page is
read in.

I don't believe this algorithm will actually work well even without concurrent
bulk access, I suspect it'll fail to work well even with something like
pgbench -S where the index fits comfortably in s_b, but the table data does
not. We already aren't good in that kind of workload, but I don't see how
your algorithm will actually achieve *any* meaningful protection for anything
but the inner index pages, the likelihood of repeat accesses to the same index
page won't be high enough to keep those pages warm.

> On the literature: you pointed at CLOCK-Pro, thanks -- the admission model
> here (probationary COOL, promote on reuse) is the 2Q-A1 / test-period idea
> and I'd rather adopt a better-justified promotion rule from that line than
> defend "second touch" if the list prefers one. I tried a stricter "promote
> only on a repeat reference" variant; it was neutral on OLTP and didn't
> improve scan resistance, so I reverted it to keep 0002 minimal, but it's an
> easy knob.

FWIW, I really doubt that any of the scan resistant algorithms actually will
allow us to get the read side strategies entirely, even with a ghost
directory. All of the scan resistant algorithms only are scan resistant if
repeated accesses (that should be cached) that occur during a concurrent bulk
scan are actually within something like the size of the buffer pool, but that
doesnt' really seem good enough.

That said, the current logic of strategies, where we actually evict the pages
with a ringbuffer, seems rather wrong, because it takes approximately forever
to fill the buffer pool even if there *are* repeat accesses.

I suspect we should go for something like an extended version of
S3-FIFO. S3-FIFO, has a FIFO for cold pages, taking up a certain percentage of
the buffer pool (10% IIRC) and a ghost directory to detect repeat accesses
that don't happen while the page is still in the cold fifo queue. That's scan
resistant as long as repeat accesses are frequent enough (1.10x the buffer
pool, I think).

To avoid bulk reads from pushing repeat accesses beyond the window in which
repeat accesses are recognized, I think we could "just" make sure the ghost
directory has eviction logic that will not allow strategy reads to take up
more than a certain percentage of the ghost directory.

Greetings,

Andres Freund