Re: EXPERIMENTAL: mmap-based memory context / allocator

On 2015-02-15 21:07:13 +0100, Tomas Vondra wrote:
> On 15.2.2015 20:56, Heikki Linnakangas wrote:
> > On 02/15/2015 08:57 PM, Tomas Vondra wrote:
> >> One of the wilder ideas (I mentined beer was involved!) was a memory
> >> allocator based on mmap [2], bypassing the libc malloc implementation
> >> altogether. mmap() has some nice features (e.g. no issues with returning
> >> memory back to the kernel, which may be problem with sbrk). So I hacked
> >> a bit and switched the AllocSet implementation to mmap().
> >
> > glibc's malloc() also uses mmap() for larger allocations. Precisely
> > because those allocations can then be handed back to the OS. I don't
> > think we'd want to use mmap() for small allocations either. Let's not
> > re-invent malloc()..
>
> malloc() does that only for allocations over MAP_THRESHOLD, which is
> 128kB by default. Vast majority of blocks we allocate are <= 8kB, so
> mmap() almost never happens.

The problem is that mmap() is, to my knowledge, noticeably more
expensive than sbrk(). Especially with concurrent workloads. Which is
why the malloc/libc authors chose to use sbrk...

IIRC glibc malloc also batches several allocation into mmap()ed areas
after some time.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services