Re: [RFC, POC] Don't require a NBuffer sized PrivateRefCount array of local buffer pins

On 8/26/14, 6:52 PM, Andres Freund wrote:
> On 2014-03-21 19:22:31 +0100, Andres Freund wrote:
>> >Hi,
>> >
>> >I've been annoyed at the amount of memory used by the backend local
>> >PrivateRefCount array for a couple of reasons:
>> >
>> >a) The performance impact of AtEOXact_Buffers() on Assert() enabled
>> > builds is really, really annoying.
>> >b) On larger nodes, the L1/2/3 cache impact of randomly accessing
>> > several megabyte big array at a high frequency is noticeable. I've
>> > seen the access to that to be the primary (yes, really) source of
>> > pipeline stalls.
>> >c) On nodes with significant shared_memory the sum of the per-backend
>> > arrays is a significant amount of memory, that could very well be
>> > used more beneficially.
>> >
>> >So what I have done in the attached proof of concept is to have a small
>> >(8 currently) array of (buffer, pincount) that's searched linearly when
>> >the refcount of a buffer is needed. When more than 8 buffers are pinned
>> >a hashtable is used to lookup the values.
>> >
>> >That seems to work fairly well. On the few tests I could run on my
>> >laptop - I've done this during a flight - it's a small performance win
>> >in all cases I could test. While saving a fair amount of memory.
> Here's the next version of this patch. The major change is that newly

<snip>

> The memory savings are clearly visible. During a pgbench scale 350, -cj
> 128 readonly run the following awk
> for pid in $(pgrep -U andres postgres); do
> grep VmData/proc/$pid/status;
> done | \
> awk 'BEGIN { sum = 0 } {sum += $2;} END { if (NR > 0) print sum/NR; else print 0;print sum;print NR}'
>
> shows:
>
> before:
> AVG: 4626.06
> TOT: 619892
> NR: 134
>
> after:
> AVG: 1610.37
> TOT: 217400
> NR: 135

These results look very encouraging, especially thinking about the cache impact. It occurs to me that it'd also be nice to have some stats available on how this is performing; perhaps a dtrace probe for whenever we overflow to the hash table, and one that shows maximum usage for a statement? (Presumably that's not much extra code or overhead...)
--
Jim C. Nasby, Data Architect jim(at)nasby(dot)net
512.569.9461 (cell) http://jim.nasby.net