Re: Changing shared_buffers without restart

Hi Ashutosh,

On Mon, 17 Aug 2026 at 17:27, Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
>
> On Fri, Jul 24, 2026 at 6:26 PM Ashutosh Bapat
> <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> >
> > Hi,
> >
> > On Fri, Feb 13, 2026 at 5:22 PM Ashutosh Bapat
> > <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> > >
> > > On Thu, Feb 12, 2026 at 7:43 PM Jakub Wartak
> > > <jakub(dot)wartak(at)enterprisedb(dot)com> wrote:
> > > >
> > > >
> > > > TBH, I haven't really looked at the code outside of that region, I'm just
> > > > trespasser that was interested in memfd ;)
> > >
> > > Your trespassing has been very helpful. I have started a separate
> > > thread to discuss resizable shared structures at [1]. Once the
> > > implementation there is somewhat finalized, it will be good to try
> > > your huge page tests again.
> >
> > Here's the next version of the patch implementing shared buffer pool
> > resizing. The patch is based on the latest master. Here's the summary
> > of changes since the last version:
> >
> > 1. The patch now uses the new shared memory infrastructure that was
> > introduced in PG 19. Patch 0006 enhances that infrastructure to
> > support resizable shared structures. I will also post the same patch
> > to [1]. I am fine to discuss the patch in that thread or here. The
> > APIs for registering and resizing the structures are documented in the
> > programming interface documentation.
>
> Palak has provided an incremental patch fixing CI failures. It needs
> to be reviewed, hence not a part of this patchset.
>
> >
> > 2. Patch 0007 implements the shared buffer pool resizing using the
> > resizable shared structures infrastructure. It has a lot of code
> > improvements, including better documentation in comments, READMEs,
> > user-facing documentation and more TAP tests. The
> > storage/buffer/README has a section on buffer resizing. The buffer
> > resizing is implemented in buf_resize.c, which also has detailed
> > comments about the implementation. I suggest starting the review with
> > the user documentation, README and buf_resize.c.
> >
>
> The attached patches have a major change in this patch: Stress tests.
> I have added one stress test for every part of the code which scans
> the buffer to stress exercise that portion of code again buffer pool
> resizing. The patch also contains fixes for the crashes or bugs
> revealed by the stress tests. Specifically the stress tests cover
> synchronization between buffer pool resizing and
> DropRelationBuffers(), DropRelationsAllBuffers(),
> DropDatabaseBuffers(), CHECKPOINT, FlushRelationBuffers(),
> FlushRelationsAllBuffers(), pg_prewarm, monitoring and diagnostic
> functions in pg_buffercache. All these tests share common utility code
> in test/buffermgr/StressUtility.pm. At the end of each stress run, it
> carries out sanity checks to make sure that the database is not
> corrupted, the shared buffer pool state is in a sane state etc.
>
> FlushDatabaseBuffers() is not covered by any stress test since it is
> only called during WAL replay of xl_dbase_create_file_copy_rec. The
> primary CREATE DATABASE and ALTER DATABASE SET TABLESPACE paths use
> RequestCheckpoint() instead. I could not find a way to reach
> FlushDatabaseBuffers through normal SQL workload. But the function
> should be able to cope with the resized buffer pool just like other
> functions which scan the buffer pool.
>
> These tests are run only when PG_TEST_EXTRA has bufmgr_stress in it
> since these tests run longer (2 minutes each) and use many resources.
> We may not want to accept all these stress tests necessarily. We may
> want to pack all of them into a single test or just not accept any of
> them. They are pretty useful to build confidence that the reisizing
> protocol, shadow variables and barriers are working correctly and are
> hazard free. I would like to keep these tests in the patchset as long
> as possible and remove them just before the final commit to keep that
> confidence as we change the code and protocol while responding to the
> review comments. We may add more deterministic white box tests, like
> 001 and 002, using injection points for specific hazardous scenarios.
>
> Following bugs/crashes were revealed by the stress tests and their
> fixes (except one) are included in the patch.
> 1. BufferSync() is fixed to clean up a buffer-invalidated-by-resize
> properly from the checkpointer datastructures.
> 2. Most of the loops scanning the buffer pool invoked CFI at the
> beginning of the loop, which meant that the buffer being processed can
> be invalidated right at the beginning of each iteration. Instead moved
> CFI calls to the end of the loop.
> 3. Fixed pg_prewarm to not rely on NBuffers being static always,
> instead it adapts to the new size after CFI. But possibly we could
> change the function to process and write one buffer's tag at a time. I
> think we need a separate discussion for this.
>
> pg_buffercache_os_pages() still crashes when it hits a concurrent
> resize. But the fix is already being written.

Attached is the fix for this. It applies on top of v20260817-0007.

Rewrites pg_buffercache_os_pages_internal() to walk one buffer at a time
with stack-local scratch arrays, so it no longer sizes anything from
NBuffers upfront. Removes the multi-call SRF machinery and switches to
InitMaterializedSRF, matching the sibling functions in the same file.

The TODO comment in 0007 ("This allocates memory using NBuffers which
may change") is addressed and removed by this patch.

015_stress_pg_buffercache now passes cleanly (653 subtests, 138s) with
concurrent resize across the six pool sizes in StressUtil.

Performance (median of 3 runs, psql \timing, no concurrent load):
pool NBuffers base numa patched numa base os_pages patched os_pages
128M 16384 34.3 37.7 27.2 19.6
512M 65536 155 170 132 98.8
2G 262144 605 658 517 385
8G 1048576 2403 2590 2076 1538

NUMA path is 8 to 10% slower due to one move_pages(2) syscall per buffer
instead of one big call for the whole pool. Non-NUMA path is 25 to 28%
faster because streaming into the tuplestore is cheaper than the old
multi-call SRF trampoline.

Thanks,
Palak