Re: dynamically allocating chunks from shared memory

On Thu, Jul 22, 2010 at 3:01 AM, Markus Wanner <markus(at)bluegap(dot)ch> wrote:
> On 07/22/2010 12:11 AM, Robert Haas wrote:
>>
>> I'm not sure why merging the SLRU pools with shared_buffers would
>> benefit from dynamically allocated shared memory.
>
> Well, I'm not sure how you'd merge SLRU pools with shared_buffers. IMO that
> inherently leads to the problem of allocating memory dynamically.
>
> With such an allocator, I'd say you just port one module after another to
> use that, instead of pre-allocated, fixed portions of shared memory.

Well, shared_buffers has to be allocated as one contiguous slab
because we index into it that way. So I don't really see how
dynamically allocating memory could help. What you'd need is a
different system for assigning buffer tags, so that a particular tag
could refer to a buffer with either kind of contents.

>> I might be at (or possibly beyond) the limit of my ability to comment
>> intelligently on this without looking more at what you want to use
>> these imessages for, but I'm still pretty skeptical about the idea of
>> storing them directly in shared memory.  It's possible, though, that I
>> am all wet.
>
> Imessages are meant to be a replacement for unix pipes. (To my knowledge,
> those don't spill to disk either, but are blocking as soon as Linux
> considers the pipe to be 'full'. Whenever that is. Or am I wrong here?)

I think you're right about that.

> The reasons for replacing them were: they consume lots of file descriptors,
> they can only be established between the parent and its child process (at
> least for anonymous pipes that's the case) and last but not least, I got
> told they still aren't fully portable. Another nice thing about imessages
> compared to unix pipes is, that it's a zero-copy approach.

That's sort of approaching the question from the opposite end from
what I was concerned about - I was wondering why you need a unicast
message-passing system.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company