Re: Question about lazy_space_alloc() / linux over-commit

On 3/7/15 12:48 AM, Noah Misch wrote:
> On Sat, Mar 07, 2015 at 12:46:42AM -0500, Tom Lane wrote:
>> Noah Misch <noah(at)leadboat(dot)com> writes:
>>> On Thu, Mar 05, 2015 at 03:28:12PM -0600, Jim Nasby wrote:
>>>> I was thinking the simpler route of just repalloc'ing... the memcpy would
>>>> suck, but much less so than the extra index pass. 64M gets us 11M tuples,
>>>> which probably isn't very common.
>>
>>> +1. Start far below 64 MiB; grow geometrically using repalloc_huge(); cap
>>> growth at vac_work_mem.
>>
>> +1 for repalloc'ing at need, but I'm not sure about the "start far below
>> 64 MiB" part. 64MB is a pretty small amount on nearly any machine these
>> days (and for anybody who thinks it isn't, that's why maintenance_work_mem
>> is a tunable).
>
> True; nothing would explode, especially since the allocation would be strictly
> smaller than it is today. However, I can't think of a place in PostgreSQL
> where a growable allocation begins so aggressively, nor a reason to break new
> ground in that respect. For comparison, tuplestore/tuplesort start memtupsize
> at 1 KiB. (One could make a separate case for that practice being wrong.)
>
>> A different line of thought is that it would seem to make sense to have
>> the initial allocation vary depending on the relation size. For instance,
>> you could assume there might be 10 dead tuples per page, and hence try to
>> alloc that much if it fits in vac_work_mem.
>
> Sounds better than a fixed 64 MiB start, though I'm not sure it's better than
> a fixed 256 KiB start.

In the case of vacuum, I think we presumably have a pretty good
indicator of how much space we should need; namely reltuples *
autovacuum_scale_factor. There shouldn't be too much more space needed
than that if autovac is keeping up with things.

If we go that route, does it still make sense to explicitly use
repalloc_huge? It will just cut over to that at some point (128M?)
anyway, and if you're vacuuming a small relation presumably it's not
worth messing with.
--
Jim Nasby, Data Architect, Blue Treble Consulting
Data in Trouble? Get it in Treble! http://BlueTreble.com