Re: our buffer replacement strategy is kind of lame

On Fri, Aug 12, 2011 at 5:05 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:

> On
> the other hand, the buffer manager has *no problem at all* trashing
> the buffer arena if we're faulting in pages for an index scan rather
> than a sequential scan.  If you manage to get all of sample_data into
> memory (by running many copies of the above query in parallel, you can
> get each one to allocate its own ring buffer, and eventually pull in
> all the pages), and then run some query that probes an index which is
> too large to fit in shared_buffers, it cheerfully blows the whole
> sample_data table out without a second thought.  Had you sequentially
> scanned a big table, of course, there would be some protection, but an
> index scan can stomp all over everything with complete impunity.

That's a good observation and I think we should do this

* Make an IndexScan use a ring buffer once it has used 32 blocks. The
vast majority won't do that, so we avoid overhead on the common path.

* Make an BitmapIndexScan use a ring buffer when we know that the
index is larger than 32 blocks. (Ignore upper parts of tree for that
calc).

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services