Re: DSA_ALLOC_NO_OOM doesn't work

On Thu, Feb 22, 2024 at 8:19 AM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
> - Separate dsm_handle, used by backend code to interact with the high
> level interface in dsm.c, from dsm_impl_handle, which is used to
> interact with the low-level functions in dsm_impl.c. This gets rid of
> the convention in dsm.c of reserving odd numbers for DSM segments stored
> in the main shmem area. There is now an explicit flag for that the
> control slot. For generating dsm_handles, we now use the same scheme we
> used to use for main-area shm segments for all DSM segments, which
> includes the slot number in the dsm_handle. The implementations use
> their own mechanisms for generating the low-level dsm_impl_handles (all
> but the SysV implementation generate a random handle and retry on
> collision).

Could we use slot number and generation number, instead of slot number
and random number? I have never liked the random number thing, which
IIUC was needed because of SysV key space management problems 'leaking'
up to the handle level (yuck). With generations, you could keep
collisions arbitrarily far apart (just decide how many bits to use).
Collisions aren't exactly likely, but if there is no need for that
approach, I'm not sure why we'd keep it. (I remember dealing with
actual collisions in the wild due to lack of PRNG seeding in workers,
which admittedly should be vanishingly rare now).

If the slot number is encoded into the handle, why do we still need a
linear search for the slot?

> - create() no longer returns the mapped_size. The old Windows
> implementation had some code to read the actual mapped size after
> creating the mapping, and returned that in *mapped_size. Others just
> returned the requested size. In principle reading the actual size might
> be useful; the caller might be able to make use of the whole mapped size
> when it's larger than requested. In practice, the callers didn't do
> that. Also, POSIX shmem on FreeBSD has similar round-up-to-page-size
> behavior but the implementation did not query the actual mapped size
> after creating the segment, so you could not rely on it.

I think that is an interesting issue with the main shmem area. There,
we can set huge_page_size to fantastically large sizes up to 16GB on
some architectures, but we have nothing to make sure we don't waste
some or most of the final page. But I agree that there's not much
point in worrying about this for DSM.

> - Added a test that exercises basic create, detach, attach functionality
> using all the different implementations supported on the current platform.

I wonder how we could test the cleanup-after-crash behaviour.