Re: introduce dynamic shared memory registry

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Nathan Bossart <nathandbossart(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: introduce dynamic shared memory registry
Date: 2023-12-08 07:36:52
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Mon, Dec 04, 2023 at 09:46:47PM -0600, Nathan Bossart wrote:
> The attached 0001 introduces a "DSM registry" to solve this problem. The
> API provides an easy way to allocate/initialize a segment or to attach to
> an existing one. The registry itself is just a dshash table that stores
> the handles keyed by a module-specified string. 0002 adds a test for the
> registry that demonstrates basic usage.
> I don't presently have any concrete plans to use this for anything, but I
> thought it might be useful for extensions for caching, etc. and wanted to
> see whether there was any interest in the feature.

Yes, tracking that in a more central way can have many usages, so your
patch sounds like a good idea. Note that we have one case in core
that be improved and make use of what you have here: autoprewarm.c.

The module supports the launch of dynamic workers but the library may
not be loaded with shared_preload_libraries, meaning that it can
allocate a chunk of shared memory worth a size of
AutoPrewarmSharedState without having requested it in a
shmem_request_hook. AutoPrewarmSharedState could be moved to a DSM
and tracked with the shared hash table introduced by the patch instead
of acquiring AddinShmemInitLock while eating the plate of other
facilities that asked for a chunk of shmem, leaving any conflict
handling to dsm_registry_table.

+dsm_registry_init_or_attach(const char *key, void **ptr, size_t size,
+ void (*init_callback) (void *ptr))

This is shaped around dshash_find_or_insert(), but it looks like you'd
want an equivalent for dshash_find(), as well.

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2023-12-08 07:51:11 Re: Rename ShmemVariableCache and initialize it in more standard way
Previous Message David Rowley 2023-12-08 07:31:53 Re: Memory consumed by paths during partitionwise join planning