Re: Better shared data structure management and resizable shared data structures

On 02/04/2026 09:58, Ashutosh Bapat wrote:
> On Wed, Apr 1, 2026 at 11:47 PM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>>> + /*
>>> + * Extra space to reserve in the shared memory segment, but it's not part
>>> + * of the struct itself. This is used for shared memory hash tables that
>>> + * can grow beyond the initial size when more buckets are allocated.
>>> + */
>>> + size_t extra_size;
>>>
>>> When we introduce resizable structures (where even the hash table
>>> directly itself could be resizable), we will introduce a new field
>>> max_size which is easy to get confused with extra_size. Maybe we can
>>> rename extra_size to something like "auxilliary_size" to mean size of
>>> the auxiliary parts of the structure which are not part of the main
>>> struct itself.
>>>
>>> + /*
>>> + * max_size is the estimated maximum number of hashtable entries. This is
>>> + * not a hard limit, but the access efficiency will degrade if it is
>>> + * exceeded substantially (since it's used to compute directory size and
>>> + * the hash table buckets will get overfull).
>>> + */
>>> + size_t max_size;
>>> +
>>> + /*
>>> + * init_size is the number of hashtable entries to preallocate. For a
>>> + * table whose maximum size is certain, this should be equal to max_size;
>>> + * that ensures that no run-time out-of-shared-memory failures can occur.
>>> + */
>>> + size_t init_size;
>>>
>>> Everytime I look at these two fields, I question whether those are the
>>> number of entries (i.e. size of the hash table) or number of bytes
>>> (size of the memory). I know it's the former, but it indicates that
>>> something needs to be changed here, like changing the names to have
>>> _entries instead of _size, or changing the type to int64 or some such.
>>> Renaming to _entries would conflict with dynahash APIs since they use
>>> _size, so maybe the latter?
>>
>> I hear you, but I didn't change these yet. If we go with the patches
>> from the "Shared hash table allocations" thread, max_size and init_size
>> will be merged into one. I'll try to settle that thread before making
>> changes here.
>
> Will review those patches next.

Those are now committed, and here's a new version rebased over those
changes. The hash options is now called 'nelems', and the 'extra_size'
in ShmemStructOpts is gone.

Plus a bunch of other fixes and cleanups. I also reordered and
re-grouped the patches a little, into more logical increments I hope.

- Heikki