Re: copy.c allocation constant

Andres Freund wrote:
> On 2018-01-24 14:25:37 -0500, Robert Haas wrote:
> > On Wed, Jan 24, 2018 at 1:43 PM, Andres Freund <andres(at)anarazel(dot)de> wrote:
> > > Indeed. Don't think RAW_BUF_SIZE is quite big enough for that on most
> > > platforms though. From man mallopt:
> > > Balancing these factors leads to a default setting of 128*1024 for the M_MMAP_THRESHOLD parameter.
> > > Additionally, even when malloc() chooses to use mmap() to back an
> > > allocation, it'll still needs a header to know the size of the
> > > allocation and such. So exactly using a size of a multiple of 4KB will
> > > still leave you with wasted space. Due to the latter I can't see it
> > > mattering whether or not we add +1 to a power-of-two size.
> >
> > Well, it depends on how it works. dsa_allocate, for example, never
> > adds a header to the size of the allocation.
>
> glibc's malloc does add a header. My half-informed suspicion is that
> most newer malloc backing allocators will have a header, because
> maintaining a shared lookup-by-address table is pretty expensive to
> maintain. A bit of metadata indicating size and/or source of the
> allocation makes using thread-local information a lot easier.

Sounds like it'd be smart to allocate something closer to
M_MMAP_THRESHOLD (which with typical values would be about double the
amount of memory the current RAW_BUF_SIZE value), minus a few dozen
bytes to allow for palloc's and malloc's respective headers. That way
we bet for a mmap()ed allocation with minimum space wastage across all
layers. Not sure whether we want to try to minimize wastage through
clever use of malloc_usable_size() on each backend's first allocation --
that'd probably just be overengineering.

--
Álvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services