Re: 2nd Level Buffer Cache

On Mon, Mar 21, 2011 at 4:47 PM, Josh Berkus <josh(at)agliodbs(dot)com> wrote:
> You're missing my point ... Postgres already *has* a 2-level cache:
> shared_buffers and the FS cache.  Anything we add to that will be adding
> levels.

I don't think those two levels are interesting -- they don't interact
cleverly at all.

I was assuming the two levels were segments of the shared buffers that
didn't interoperate at all. If you kick buffers from the higher level
cache into the lower level one then why not just increase the number
of clock sweeps before you flush a buffer and insert non-index pages
into a lower clock level instead of writing code for two levels?

I don't think it will outperform in general because LRU is provably
within some margin from optimal and the clock sweep is an approximate
LRU. The only place you're going to find wins is when you know
something extra about the *future* access pattern that the lru/clock
doesn't know based on the past behaviour. Just saying "indexes are
heavily used" or "system tables are heavily used" isn't really extra
information since the LRU can figure that out. Something like
"sequential scans of tables larger than shared buffers don't go back
and read old pages before they age out" is.

The other place you might win is if you have some queries that you
want to always be fast at the expense of slower queries. So your short
web queries that only need to touch a few small tables and system
tables can tag buffers that are higher priority and shouldn't be
swapped out to achieve a slightly higher hit rate on the global cache.

--
greg