Re: Better shared data structure management and resizable shared data structures

On Tue, Apr 7, 2026 at 3:36 PM Ashutosh Bapat
<ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
>
> On Mon, Apr 6, 2026 at 7:23 PM Ashutosh Bapat
> <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> wrote:
> >
> > I have kept these two patches separate from the main patch so that I
> > can remove them if others feel they are not worth including in the
> > feature.
>
> Here are patches rebased on the latest HEAD. No conflicts just rebase.
>
> Here are differences from the previous patchset.
>
> o. There are two patches in this patchset now. a. 0001 which supports
> resizable shared memory and is equivalent to 0001 + 0002 + 0004 + 0005
> from the previous patchset. b. 0002 which is 0006 from the previous
> patchset and adds support for protecting resizable shared memory
> structures. 0003, which added diagnostics to investigate CFBot
> failure, from the previous patchset is not required anymore since all
> tests pass with CFBot.
>
> o. I have merged 0002 into 0001 from the previous patchset since with
> that patch all platforms are green on CFBot. The resizable shared
> memory test now uses /proc/self/smaps instead of /proc/self/status to
> find the amount of memory allocated in the main shared memory segment
> of PostgreSQL.
>
> o. Merged 0004, which supported minimum_size, into 0001. Minimum_size
> would be useful to protect against accidental shrinkage of the
> resizable structures. It will help additional support for minimum
> sizes of GUCs like shared_buffers. It also makes it easy and intuitive
> to distinguish between fixed-size and resizable structures, and will
> be useful to find the minimum size of the shared memory segment.
>
> o. Merged 0005, which allows ABI compatibility between the binaries
> which support resizable shared memory and those which don't, into
> 0001. Apart from ABI compatibility, the code has lesser #ifdef blocks
> and thus easier to read and maintain.
>
> I didn't find it useful to keep 0004 and 0005 separate since they were
> interdependent and made review complicated and have higher chances of
> being acceptable.
>
> o. 0006 is still separate since I am not sure whether the
> functionality is absolutely needed at this time. In an offlist
> discussion, Andres mentioned that it is not strictly needed. The
> subsystem that uses the resizable shared memory can implement their
> own protection if required and integrate it in the subsystems specific
> synchronization. But Matthias thinks different. The API to add
> protection is platform dependent, so it's better to abstract it via
> shmem.c. If we decide to accept this patch, we should merge it into
> 0001 before committing.
>
> Also did some more cleanups and changed the name of the GUC
> have_resizable_shmem to have_resizable_shared_memory since shmem is an
> internal phrase.
>
> I am looking at merging the resizable_shmem module into test_shmem module next.

Here are patches with the test modules merged.

The merged module looks a bit rough to me and so does 0006. For
example, I am not sure whether calling ShmemStructProtect() from
init_fn is a good idea. See [1] for example. But init_fn is the last
chance for the subsystem to touch and setup the resizable structure
before it's opened to the wild. So, in the current infrastructure, I
don't see any better place to call ShmemStructProtect() either. If you
run tests after applying patch 0006, you will need to apply patch
attached to [1] as well; otherwise the test will hang.

[1] https://www.postgresql.org/message-id/CAExHW5uMQGvQH6GKaBZVtH4S9O13TwN+_0Vy1gUpAW=_T_AmRA@mail.gmail.com

--
Best Wishes,
Ashutosh Bapat