Re: Question about lazy_space_alloc() / linux over-commit

On Sat, Mar 7, 2015 at 5:49 PM, Andres Freund <andres(at)2ndquadrant(dot)com> wrote:
> On 2015-03-05 15:28:12 -0600, Jim Nasby wrote:
>> I was thinking the simpler route of just repalloc'ing... the memcpy would
>> suck, but much less so than the extra index pass. 64M gets us 11M tuples,
>> which probably isn't very common.
>
> That has the chance of considerably increasing the peak memory usage
> though, as you obviously need both the old and new allocation during the
> repalloc().
>
> And in contrast to the unused memory at the tail of the array, which
> will usually not be actually allocated by the OS at all, this is memory
> that's actually read/written respectively.

Yeah, I'm not sure why everybody wants to repalloc() that instead of
making several separate allocations as needed. That would avoid
increasing peak memory usage, and would avoid any risk of needing to
copy the whole array. Also, you could grow in smaller chunks, like
64MB at a time instead of 1GB or more at a time. Doubling an
allocation that's already 1GB or more gets big in a hurry.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company