Skip site navigation (1) Skip section navigation (2)

Autotuning of shared buffer size (was: Re: Getting rid of AtEOXact Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...))

From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Cc: simon(at)2ndquadrant(dot)com, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>,josh(at)agliodbs(dot)com, pgsql-hackers(at)postgresql(dot)org,pgsql-performance(at)postgresql(dot)org
Subject: Autotuning of shared buffer size (was: Re: Getting rid of AtEOXact Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...))
Date: 2004-10-18 21:19:17
Message-ID: 41743355.9060005@Yahoo.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
Trying to think a little out of the box, how "common" is it in modern 
operating systems to be able to swap out shared memory?

Maybe we're not using the ARC algorithm correctly after all. The ARC 
algorithm does not consider the second level OS buffer cache in it's 
design. Maybe the total size of the ARC cache directory should not be 2x 
the size of what is configured as the shared buffer cache, but rather 2x 
the size of the effective cache size (in 8k pages). If we assume that 
the job of the T1 queue is better done by the OS buffers anyway (and 
this is what this discussion seems to point out), we shouldn't hold them 
in shared buffers (only very few of them and evict them ASAP). We just 
account for them and assume that the OS will have those cached that we 
find in our T1 directory. I think with the right configuration for 
effective cache size, this is a fair assumption. The T2 queue represents 
the frequently used blocks. If our implementation would cause the 
unused/used portions of the buffers not to move around, the OS will swap 
out currently unused portions of the shared buffer cache and utilize 
those as OS buffers.

To verify this theory it would be interesting what the ARC strategy 
after a long DBT run with a "large" buffer cache thinks a good T2 size 
would be. Enabling the strategy debug message and running postmaster 
with -d1 will show that. In theory, this size should be anywhere near 
the sweet spot.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck(at)Yahoo(dot)com #

In response to

pgsql-performance by date

Next:From: Tom LaneDate: 2004-10-19 00:02:03
Subject: Re: Indexes performance
Previous:From: Tom LaneDate: 2004-10-18 21:01:25
Subject: Re: Getting rid of AtEOXact Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...)

pgsql-hackers by date

Next:From: Tom LaneDate: 2004-10-18 22:03:50
Subject: Re: [HACKERS] Final libpq patch?
Previous:From: Tom LaneDate: 2004-10-18 21:01:25
Subject: Re: Getting rid of AtEOXact Buffers (was Re: [Testperf-general] Re: [PERFORM] First set of OSDL Shared Memscalability results, some wierdness ...)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group