Re: patch for new feature: Buffer Cache Hibernation

On Fri, May 6, 2011 at 5:31 PM, Greg Smith <greg(at)2ndquadrant(dot)com> wrote:
> On 05/05/2011 05:06 AM, Mitsuru IWASAKI wrote:
>>
>> In summary, PgFincore's target is File System Buffer Cache, Buffer
>> Cache Hibernation's target is DB Buffer Cache(shared buffers).
>>
>
> Right.  The thing to realize is that shared_buffers is becoming a smaller
> fraction of the total RAM used by the database every year.  On Windows it's
> been stuck at useful settings being less than 512MB for a while now.  And on
> UNIX systems, around 8GB seems to be effective upper limit.  Best case,
> shared_buffers is only going to be around 25% of total RAM; worst-case,
> approximately, you might have Windows server with 64GB of RAM where
> shared_buffers is less than 1% of total RAM.
>
> There's nothing wrong with the general idea you're suggesting.  It's just
> only targeting a small (and shrinking) subset of the real problem here.
>  Rebuilding cache state starts with shared_buffers, but that's not enough of
> the problem to be an effective tweak on many systems.
>
> I think that all the complexity with CRCs etc. is unlikely to lead anywhere
> too, and those two issues are not completely unrelated.  The simplest,
> safest thing here is the right way to approach this, not the most
> complicated one, and a simpler format might add some flexibility here to
> reload more cache state too.  The bottleneck on reloading the cache state is
> reading everything from disk.  Trying to micro-optimize any other part of
> that is moving in the wrong direction to me.  I doubt you'll ever measure a
> useful benefit that overcomes the expense of maintaining the code.  And you
> seem to be moving to where someone can't restore cache state when they
> change shared_buffers.  A simpler implementation might still work in that
> situation; reload until you run out of buffers if shared_buffers shrinks,
> reload until you're done with the original size.

Yeah, I'm pretty well convinced this whole approach is a dead end.
Priming the OS buffer cache seems way more useful. I also think
saving the blocks to be read rather than the actual blocks makes a lot
more sense.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company