Re: [PATCHES] scan_recycle_buffers

On Sat, 2007-03-10 at 09:42 +0000, Gregory Stark wrote:
> "Simon Riggs" <simon(at)2ndquadrant(dot)com> writes:
>
> > COPY command now also uses this hint, to allow test results and
> > discussion. Others could also, perhaps needing different values.
>
> Hm. It occurs to me that different commands may want different size buffer
> rings.

Yes, thats noted in comments in the patch. scan_recycle_buffers was
designed to allow us to test which types of scan benefit from which
settings.

> As I understand it the reason your buffer rings are more than just a single
> target buffer are:
>
> 1) For sequential scans because it creates a window for synchronized
> sequential scans.
>
> 2) For dirty buffers because the dirty evicting the dirty buffer will force an
> XLogFlush and we want to give a chance for the WAL pointer to advance past
> the buffer's LSN. Ie, to allow other transactions to do our fsync for us
> since it won't cost them much extra if anything.
>
> Can you log whenever your ring buffer finds a provides a dirty buffer whose
> LSN requires syncing the WAL log? That will help you figure out how large a
> ring buffer you need to guarantee property 2.

Hmm, again your thoughts mirrored my own, but this time you're slightly
ahead of me. I was just looking into the possibility of adaptive scans,
to allow synch scans to force the scan_recycle_buffer size higher. I
think having the size of the buffer vary during a scan seems sensible
also, within min and max limits.

I'll post some further thoughts tomorrow.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com