Re: PATCH: two slab-like memory allocators

On 10/02/2016 12:23 AM, John Gorman wrote:
> I reproduced the quadradic pfree performance problem and verified that
> these patches solved it.
>
> The slab.c data structures and functions contain no quadradic components.
>
> I noticed the sizing loop in SlabContextCreate() and came up with
> a similar formula to determine chunksPerBlock that you arrived at.
>
>> Firstly, I've realized there's an issue when chunkSize gets too
>> large - once it exceeds blockSize, the SlabContextCreate() fails
>> as it's impossible to place a single chunk into the block. In
>> reorderbuffer, this may happen when the tuples (allocated in
>> tup_context) get larger than 8MB, as the context uses
>> SLAB_LARGE_BLOCK_SIZE (which is 8MB).
>>
>> But maybe there's a simpler solution - we may simply cap the
>> chunkSize (in GenSlab) to ALLOC_CHUNK_LIMIT. That's fine, because
>> AllocSet handles those requests in a special way - for example
>> instead of tracking them in freelist, those chunks got freed
>> immediately.
>
> I like this approach because it fixes the performance problems
> with smaller allocations and doesn't change how larger
> allocations are handled.
>

One more comment about GenSlab, particularly about unpredictability of
the repalloc() behavior. It works for "large" chunks allocated in the
AllocSet part, and mostly does not work for "small" chunks allocated in
the SlabContext. Moreover, the chunkSize changes over time, so for two
chunks of the same size, repalloc() may work on one of them and fail on
the other, depending on time of allocation.

With SlabContext it's perfectly predictable - repalloc() call fails
unless the chunk size is exactly the same as before (which is perhaps a
bit pointless, but if we decide to fail even in this case it'll work
100% time).

But with GenSlabContext it's unclear whether the call fails or not.

I don't like this unpredictability - I'd much rather have consistent
failures (making sure people don't do repalloc() on with GenSlab). But I
don't see a nice way to achieve that - the repalloc() call does not go
through GenSlabRealloc() at all, but directly to SlabRealloc() or
AllocSetRealloc().

The best solution I can think of is adding an alternate version of
AllocSetMethods, pointing to a different AllocSetReset implementation.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services