Re: PATCH: two slab-like memory allocators

From: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>, Petr Jelinek <petr(at)2ndquadrant(dot)com>
Cc: John Gorman <johngorman2(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: PATCH: two slab-like memory allocators
Date: 2016-10-23 14:26:28
Message-ID: 34a6d4ea-7f3d-a62c-fff1-7a9108c078d7@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On 10/22/2016 08:30 PM, Tomas Vondra wrote:
> On 10/20/2016 04:43 PM, Robert Haas wrote:
>>
>> ...
>>
>> The sb_alloc allocator I proposed a couple of years ago would work
>> well for this case, I think.
>>
>
> Maybe, but it does not follow the Memory Context design at all, if I
> understand it correctly. I was willing to give it a spin anyway and see
> how it compares to the two other allocators, but this is a significant
> paradigm shift and certainly much larger step than what I proposed.
>
> I'm not even sure it's possible to implement a MemoryContext based on
> the same ideas as sb_alloc(), because one of the important points of
> sb_alloc design seems to be throwing away the chunk header. While that
> may be possible, it would certainly affect the whole tree (not just the
> reorderbuffer bit), and it'd require way more work.
>
> Moreover, the two allocators I proposed significantly benefit from the
> "same lifespan" assumption. I don't think sb_alloc can do that.
>

I've given the sb_alloc patch another try - essentially hacking it into
reorderbuffer, ignoring the issues mentioned yesterday. And yes, it's
faster than the allocators discussed in this thread. Based on a few very
quick tests on my laptop, the difference is usually ~5-10%.

That might seem like a significant improvement, but it's negligible
compared to the "master -> slab/gen" improvement, which improves
performance by orders of magnitude (at least for the tested cases).

Moreover, the slab/gen allocators proposed here seem like a better fit
for reorderbuffer, e.g. because they release memory. I haven't looked at
sb_alloc too closely, but I think it behaves more like AllocSet in this
regard (i.e. keeping the memory indefinitely).

FWIW I'm not making any conclusions about sb_alloc benefits outside
reorderbuffer.c - it might easily be worth pursuing, no doubt about
that. The amount of remaining work however seems quite high, though.

Attached is the modified sb_alloc patch that I used - it's mostly v1
with removed uses in nbtree etc. FWIW the patch does not implement
sb_destroy_private_allocator (it's only defined in the header), which
seems like a bug.

regards

--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
sballoc-v2-tomas.patch text/x-diff 137.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Guillaume Lelarge 2016-10-23 15:06:57 Re: Exclude pg_largeobject form pg_dump
Previous Message Magnus Hagander 2016-10-23 13:57:39 pgsql: Remove extra comma at end of enum list