Re: EXPERIMENTAL: mmap-based memory context / allocator

Andres Freund <andres(at)2ndquadrant(dot)com> writes:
> On 2015-02-15 21:07:13 +0100, Tomas Vondra wrote:
>> On 15.2.2015 20:56, Heikki Linnakangas wrote:
>>> glibc's malloc() also uses mmap() for larger allocations. Precisely
>>> because those allocations can then be handed back to the OS. I don't
>>> think we'd want to use mmap() for small allocations either. Let's not
>>> re-invent malloc()..

>> malloc() does that only for allocations over MAP_THRESHOLD, which is
>> 128kB by default. Vast majority of blocks we allocate are <= 8kB, so
>> mmap() almost never happens.

> The problem is that mmap() is, to my knowledge, noticeably more
> expensive than sbrk(). Especially with concurrent workloads. Which is
> why the malloc/libc authors chose to use sbrk...

> IIRC glibc malloc also batches several allocation into mmap()ed areas
> after some time.

Keep in mind also that aset.c doubles the request size every time it goes
back to malloc() for some more space for a given context. So you get up
to 128kB pretty quickly.

There will be a population of 8K-to-64K chunks that don't ever get
returned to the OS but float back and forth between different
MemoryContexts as those are created and deleted. I'm inclined to think
this is fine and we don't need to improve on it.

Part of the reason for my optimism is that on glibc-based platforms,
IME PG backends do pretty well at reducing their memory consumption back
down to a minimal value after each query. (On other platforms, not so
much, but arguably that's libc's fault not ours.) So I'm not really
seeing a problem that needs fixed, and definitely not one that a
platform-specific fix will do much for.

regards, tom lane