Re: dynamically allocating chunks from shared memory

Hi,

On 08/09/2010 06:31 PM, Bruce Momjian wrote:
> Let me be more concrete. Suppose you are using threads, and you want to
> increase your shared memory from 20MB to 30MB. How do you do that?

There's absolutely no need to pre-allocate 20 MB in advance in a
threaded environment. You just allocate memory in small chunks. For a
threaded-model, that memory is shared by default, so the total amount of
shared memory can grow and shrink very easily. (And even makes usused
memory available to other processes, not just other threads).

> If
> you want it contiguous, you have to use realloc, which might move the
> pointer. If you allocate another 10MB chunk, you then have shared
> memory fragments, which is the same as adding another shared memory
> segment.

Okay, I think I now understand the requirement of continuity you
mentioned earlier already. I agree that with the current approach, we
cannot simply use such a dynamic allocator to solve all of our problems.

Every subsystem would need to be converted to something that allocates
shared memory in smaller chunks for such a dynamic allocator to be of
any use. Robert already pointed out that this may be troublesome for
shared_buffers, which is by far the largest consumer of shared memory. I
didn't look into this, yet. And I'd like to hear more about the
feasibility of that approach for other subsystems.

Another issue to be discussed would be the limits of sharing free memory
between subsystems. Maybe we even reach the conclusion that we
absolutely *want* fixed maximum sizes for every single subsystem so as
to be able to guarantee a certain amount of multi-xact or SLRU entries
at any point in time (otherwise one memory hungry subsystem could
possibly eat it all up with another subsystem getting the OOM error when
trying to allocate for its very first entry).

Thanks for bringing this discussion to live again.

Regards

Markus Wanner