Re: Group commit, revised

On 17.01.2012 16:35, Peter Geoghegan wrote:
> On 16 January 2012 08:11, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> I think it might be simpler if it wasn't the background writer that's
>> responsible for "driving" the group commit queue, but the backends
>> themselves. When a flush request comes in, you join the queue, and if
>> someone else is already doing the flush, sleep until the driver wakes you
>> up. If no-one is doing the flush yet (ie. the queue is empty), start doing
>> it yourself. You'll need a state variable to keep track who's driving the
>> queue, but otherwise I think it would be simpler as there would be no
>> dependency on WAL writer.
>
> I think this replaces one problem with another. You've now effectively
> elevated a nominated backend to the status of an auxiliary process -
> do you intend to have the postmaster look after it, as with any other
> auxiliary process?

The GroupCommitWaitForLSN() call happens within a critical section. If
the process dies, you're screwed no matter what. It's not very different
from the current situation where if one backend flushes the WAL, another
backend will notice that once it gets the WALWriteLock, and returns quickly.

>> wal_writer_delay is still needed for controlling how often asynchronous
>> commits are flushed to disk.
>
> That had occurred to me of course, but has anyone ever actually
> tweaked wal_writer_delay with adjusting the behaviour of asynchronous
> commits in mind?

I found it very helpful to reduce wal_writer_delay in pgbench tests,
when running with synchronous_commit=off. The reason is that hint bits
don't get set until the commit record is flushed to disk, so making the
flushes more frequent reduces the contention on the clog. However, Simon
made async commits nudge WAL writer if the WAL page fills up, so I'm not
sure how relevant that experience is anymore.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com