RE: Changing shared_buffers without restart

From: Jack Ng <Jack(dot)Ng(at)huawei(dot)com>
To: Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
Cc: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Ni Ku <jakkuniku(at)gmail(dot)com>
Subject: RE: Changing shared_buffers without restart
Date: 2025-05-07 05:34:37
Message-ID: dcd396b2e89c4b83a34adaaec5838476@huawei.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> all the possible scenarios. But now I'm reworking it along the lines suggested
> by Thomas, and will address those as well. Thanks!

Thanks for the info, Dmitry.
Just want to confirm my understanding of Thomas' suggestion and your discussions... I think the simpler and more portable solution goes something like the following?

* For each BP resource segment (main, desc, buffers, etc):
1. create an anonymous file as backing
2. mmap a large reserved shared memory area with PROTO_READ/WRITE + MAP_NORESERVE using the anon fd
3. use ftruncate to back the in-use region (and maybe posix_fallocate too to avoid SIGBUS on alloc failure during first-touch), but no need to create a memory mapping for it
4. also no need to create a separate mapping for the reserved region (already covered by the mapping created in 2.)

|-- Memory mapping (MAP_NORESERVE) for BUFFER --|
|-- In-use region --|----- Reserved region -----|

* During resize, simply calculate the new size and call ftruncate on each segment to adjust memory accordingly, no need to mmap/munmap or modify any memory mapping.

I tried this approach with a test program (with huge pages), and both expand and shrink seem to work as expected --for shrink, the memory is freed right after the resize ftruncate.

Regards,

Jack Ng

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2025-05-07 05:56:56 Re: minor fix related to Auxiliary processes and IO workers
Previous Message Andrei Lepikhov 2025-05-07 05:02:05 Re: MergeAppend could consider sorting cheapest child path