Re: Experimental dynamic memory allocation of postgresql shared memory

On Fri, Jun 17, 2016 at 9:30 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Aleksey Demakov <ademakov(at)gmail(dot)com> writes:
>> I have some very experimental code to enable dynamic memory allocation
>> of shared memory for postgresql backend processes.
>
> Um ... what's this do that the existing DSM stuff doesn't do?
>

It operates over a single large shared memory segment. Within this
segment it lets alloc / free small chunks of memory from 16 bytes to
16 kilobytes. Chunks are carved out from fixed-size 32k blocks. Each
block is used to allocate chunks of single size class. When a block is
full, another block for a given size class is taken from the top
shared segment.

The goal is to support high levels of concurrency for alloc / free
calls. Therefore the allocator is mostly non-blocking. Currently it
uses Heller's lazy list algorithm to maintain block lists of a given
size class, so it uses slocks once in a while, when a new block is
added or removed. If this proves to cause scalability problems the
Heller's list might be replaced with Maged Michael's lock-free list to
make the whole allocator absolutely lock-free.

Additionally it provides epoch-based memory reclamation facility that
solves ABA-problem for lock-free algorithms. I am going to implement
some lock-free algorithms (extendable hash-tables and probably skip
lists) on top of this facility.

Regards,
Aleksey