Re: Buffer locking is special (hints, checksums, AIO writes)

From: Andres Freund <andres(at)anarazel(dot)de>
To: Melanie Plageman <melanieplageman(at)gmail(dot)com>
Cc: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, pgsql-hackers(at)postgresql(dot)org, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Noah Misch <noah(at)leadboat(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Subject: Re: Buffer locking is special (hints, checksums, AIO writes)
Date: 2025-12-03 00:47:35
Message-ID: lneuyxqxamqoayd2ntau3lqjblzdckw6tjgeu4574ezwh4tzlg@noioxkquezdw
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On 2025-11-25 11:54:00 -0500, Andres Freund wrote:
> Thanks a lot for that detailed review! A few questions and comments, before I
> try to address the comments in the next version.

Here's that new new version, with the following changes

- Some more micro-optimizations, most importantly adding a commit that doesn't
initialize the delay in LockBufHdr() unless needed. With those I don't see a
consistent slowdown anymore (slight speedup on one workstation, slight
slowdown on another, in an absurdly adverse workload)

- Tried to address Melanie's feedback, with some exceptions (some noted below,
but I also need to make another pass through the reviews)

- re-implemented AssertNotCatalogBufferLock() in the new world

- Substantially expanded comments around setting hint bits (in buffer/README,
heapam_visibility.c and bufmgr.c)

- split out the change to fsm_vacuum_page() to start to lock the page into is
own commit

- reordered patch series so that smaller changes are before the 64bit-state
and "Implement buffer content locks independently of" commits, so they can
be committed while we finish cleaning the later changes

- I didn't invest much in cleaning up the later patches ("Don't copy pages
while writing out" and "Make UnlockReleaseBuffer() more efficient") yet,
wanted to focus on the earlier patches first

Todo:

- still need to rename ResOwnerReleaseBufferPin(). Wondering about what to
rename ResourceOwnerDesc.name to. "buffer ownership" maybe? Not great...

- gistkillitems() complaint by Melanie

- amortize vs batch vs SetHintBits comment + SHB_* names

- for the next version I'll remove the BATCHMVCC_FEWER_ARGS conditionals from
0010. I don't love needing BatchMVCCState but I don't really see an
alternative, the performance difference is pretty persistent.

Questions:
- ForEachLWLockHeldByMe() and LWLockDisown() aren't used anymore, should we
remove them?

Greetings,

Andres Freund

Attachment Content-Type Size
v7-0001-bufmgr-Turn-BUFFER_LOCK_-into-an-enum.patch text/x-diff 3.2 KB
v7-0002-Add-pg_atomic_unlocked_write_u64.patch text/x-diff 2.0 KB
v7-0003-Rename-BUFFERPIN-wait-event-class-to-BUFFER.patch text/x-diff 6.6 KB
v7-0004-bufmgr-Optimize-LockBufHdr-by-delaying-spin-delay.patch text/x-diff 2.3 KB
v7-0005-bufmgr-Separate-keys-for-private-refcount-infrast.patch text/x-diff 11.2 KB
v7-0006-bufmgr-Add-one-entry-cache-for-private-refcount.patch text/x-diff 4.9 KB
v7-0007-freespace-Don-t-modify-page-without-any-lock.patch text/x-diff 2.0 KB
v7-0008-heapam-Move-logic-to-handle-HEAP_MOVED-into-a-hel.patch text/x-diff 11.5 KB
v7-0009-heapam-Use-exclusive-lock-on-old-page-in-CLUSTER.patch text/x-diff 2.7 KB
v7-0010-heapam-Add-batch-mode-mvcc-check-and-use-it-in-pa.patch text/x-diff 7.6 KB
v7-0011-bufmgr-Change-BufferDesc.state-to-be-a-64bit-atom.patch text/x-diff 46.6 KB
v7-0012-bufmgr-Implement-buffer-content-locks-independent.patch text/x-diff 44.6 KB
v7-0013-Require-share-exclusive-lock-to-set-hint-bits-and.patch text/x-diff 37.9 KB
v7-0014-WIP-Make-UnlockReleaseBuffer-more-efficient.patch text/x-diff 3.5 KB
v7-0015-WIP-bufmgr-Don-t-copy-pages-while-writing-out.patch text/x-diff 11.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2025-12-03 00:47:56 Re: PG version is not seen in pg_upgrade test log
Previous Message Tom Lane 2025-12-03 00:40:46 Re: Consistently use palloc_object() and palloc_array()