Re: Better shared data structure management and resizable shared data structures

On 12/03/2026 20:56, Robert Haas wrote:
> On Thu, Mar 12, 2026 at 2:41 PM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>> shmem_startup_hook() is too late. The shmem structs need to be
>> registered at postmaster startup before the shmem segment is allocated,
>> so that we can calculate the total size needed.
>
> Sorry, I meant shmem_request_hook.

Ah ok

>> I'm currently leaning towards _PG_init(), except for allocations that
>> depend on MaxBackends. For those, you can install a shmem_request_hook
>> that sets the size in the descriptor. In other words, you can leave the
>> 'size' as empty in _PG_init(), but set it later in the shmem_request_hook.
>
> Why can't you just do the whole thing later?

shmem_request_hook won't work in EXEC_BACKEND mode, because in
EXEC_BACKEND mode, ShmemRegisterStruct() also needs to be called at
backend startup.

One of my design goals is to avoid EXEC_BACKEND specific steps so that
if you write your extension oblivious to EXEC_BACKEND mode, it will
still usually work with EXEC_BACKEND. For example, if it was necessary
to call a separate AttachShmem() function for every shmem struct in
EXEC_BACKEND mode, but which was not needed on Unix, that would be bad.

>> Except that you'd still need them for RequestNamedLWLockTranche(). I
>> wonder if we should recommend extensions to embed the LWLock struct into
>> their shared memory struct and use the LWLockInitialize() and
>> LWLockNewTrancheId() functions instead. That fits the new
>> ShmemRegisterStruct() API a little better than RequestNamedLWLockTranche().
>
> Yeah, I think RequestNamedLWLockTranche() might be fine if you just
> need LWLocks, but if you need a bunch of resources, putting them all
> into the same chunk of memory seems cleaner.

Agreed. Then again, how often do you need just a LWLock (or multiple
LWLocks)? Surely you have a struct you want to protect with the lock. I
guess having shmem hash table but no struct would be pretty common, though.

- Heikki