Re: BUG #14231: logical replication wal sender process spins when using error traps in function

On 07/06/2016 11:40 PM, Peter Geoghegan wrote:
> On Wed, Jul 6, 2016 at 2:26 PM, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com> wrote:
>>> So I think this is primarily an artifact of doing so much retail
>>> palloc/pfree in a single memory context.
>>
>> As I recall, this is the main reason Andres stuck the slab cache in
>> reorderbuffer. Maybe it'd be worthwhile to implement a different
>> MemoryContext tailored to this use case, and remove the slab cache
>> stuff.
>
> I withdraw my remarks on that commit -- I don't think that's what it is now.
>
> I had luck adding a new child memory context for tuplesort.c in commit
> 0011c0091e886b874e485a46ff2c94222ffbf550. This allowed tuplesort.c to
> blow away the free list via a new call to MemoryContextReset(). This
> didn't even remove *any* pfree() operations in the affected codepath,
> but was still quite effective.
>
> I'm not sure that that's comparable, but it may be an interesting case
> that others are not aware of.
>

I'm not very familiar with the reorderbuffer.c, but my understanding is
that it's an object with potentially very long lifetime, doing a lot of
interleaved palloc/pfree (or accessing the slab cache). The lifetime of
tuplesort is much shorter with clear boundaries, and probably does not
need palloc/pfree interleaved that much.

The separate memory context would probably fix the pfree issue, but I'm
afraid it'd make all pfree() calls pointless useless as AllocSet keeps
all pfreed chunks in internal freelists and only releases them at the
end. Which means the memory context would allocate memory and then keep
it until it's destroyed, meaning the slab limits would be pointless.

I do have a few rough ideas how this might be improved, but it's a bit
rough at this point. The distinguishing feature of the slab allocator is
that all objects have exactly the same size, which makes the "many
freelist" idea in AllocSet a bit pointless. (Let's ignore that we use
the same context for three different objects for now.)

So I'm thinking about adding a new MemoryContext implementation, e.g.
SlabMemoryContext, which will only allow allocating pieces with a
particular size. Then we can get rid of the AllocSet pieces that deal
with freelists, and also make sure we know which blocks contain free
chunks so that we can reuse chunks from more full blocks, etc. But I
haven't tried coding anything, so maybe it's all stupid.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services