Re: SLRUs in the main buffer pool - Page Header definitions

Greetings,

* Robert Haas (robertmhaas(at)gmail(dot)com) wrote:
> On Thu, Aug 24, 2023 at 3:28 PM Stephen Frost <sfrost(at)snowman(dot)net> wrote:
> > Agreed that we'd certainly want to make sure it's all correct and to do
> > performance testing but in terms of how many buffers... isn't much of
> > the point of this that we have data in memory because it's being used
> > and if it's not then we evict it? That is, I wouldn't think we'd have
> > set parts of the buffer pool for SLRUs vs. regular data but would let
> > the actual demand drive what pages are in the cache and I thought that
> > was part of this proposal and part of the reasoning behind making this
> > change.
>
> I think that it's not quite that simple. In the regular buffer pool,
> access to pages is controlled by buffer pins and buffer content locks,
> but these mechanisms don't exist in the same way in the SLRU code. But
> buffer pins drive usage counts which drive eviction decisions. So if
> you move SLRU data into the main buffer pool, you either need to keep
> the current locking regime and use some new logic to decide how much
> of shared_buffers to bequeath to the SLRU pools, OR you need to make
> SLRU access use buffer pins and buffer content locks. If you do the
> latter, I think you substantially increase the cost of an uncontended
> SLRU buffer access, because you now need to pin the buffer, and and
> then take and release the content lock, and then release the pin;
> whereas today you can do all that by just taking and release the
> SLRU's lwlock. That's more atomic operations, and hence more costly, I
> think. But even if not, it could perform terribly if SLRU buffers
> become more vulnerable to eviction than they are at present, or
> alternatively if they take over too much of the buffer pool and force
> other important data out.

An SLRU buffer access does also update the cur_lru_count for the SLRU,
along with the per-page page_lru_count, but those are 32bit and we don't
enforce that they're done in order, so presumably those are less
expensive than the pinning and usage count updates.

This thread started with the issue that our current SLRUs are relatively
small though and simply increasing their size would lead to issues as
we're just doing simple things like a linear search through them all at
times, or at least that's more-or-less what I understood from [1]. More
details on the specific 'scaling and sizing challenges' would be nice to
have. The current patches were at least claimed to improve performance
while also using ReadBuffer_common [2]. Having an idea as to what is
specifically leading to that would be interesting though with all these
changes likely non-trivial. pgbench may not be the best way to measure
this, but it's still interesting to see an improvement like that.

Certainly one concern about using the regular buffer pool is that
finding a victim page can be expensive and having that as part of an
SLRU access could be pretty painful. Though we also have that problem
elsewhere too.

If we're going to effectively segregate the buffer pool into SLRU parts
vs. everything else and then use the existing strategies for SLRUs and
have that be different from what everything else is using ... then
that doesn't seem like it's really changing things. What would be the
point of moving the SLRUs into the main buffer pool then?

> SLRUs are a performance hotspot, so even relatively minor changes to
> their performance characteristics can, I believe, have pretty
> noticeable effects on performance overall.

Agreed, we certainly need to have a plan for how to performance test
this change and should try to come up with some 'worst case' tests.

Thanks,

Stephen

[1]: https://postgr.es/m/EFAAC0BE-27E9-4186-B925-79B7C696D5AC%40amazon.com
[2]: https://postgr.es/m/A09EAE0D-0D3F-4A34-ADE9-8AC1DCBE7D57%40amazon.com