Re: Experimental patch for inter-page delay in VACUUM

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Jan Wieck <JanWieck(at)Yahoo(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Greg Stark <gsstark(at)mit(dot)edu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Experimental patch for inter-page delay in VACUUM
Date: 2003-11-10 18:57:01
Message-ID: 200311101857.hAAIv1223046@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Jan Wieck wrote:
> > If the background cleaner has to not just write() but write/fsync or
> > write/O_SYNC, it isn't going to be able to clean them fast enough. It
> > creates a bottleneck where we didn't have one before.
> >
> > We are trying to eliminate an I/O storm during checkpoint, but the
> > solutions seem to be making the non-checkpoint times slower.
> >
>
> It looks as if you're assuming that I am making the backends unable to
> write on their own, so that they have to wait on the checkpointer. I
> never said that.
>
> If the checkpointer keeps the LRU heads clean, that lifts off write load
> from the backends. Sure, they will be able to dirty pages faster.
> Theoretically, because in practice if you have a reasonably good cache
> hitrate, they will just find already dirty buffers where they just add
> some more dust.
>
> If after all the checkpointer (doing write()+whateversync) is not able
> to keep up with the speed of buffers getting dirtied, the backends will
> have to do some write()'s again, because they will eat up the clean
> buffers at the LRU head and pass the checkpointer.

Yes, there are a couple of issues here --- first, have a background
writer to write dirty pages. This is good, no question. The bigger
question is removing sync() and using fsync() or O_SYNC for every write
--- if we do that, the backends doing private write will have to fsync
their writes too, meaning if the checkpointer can't keep up, we now have
backends doing slow writes too.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2003-11-10 18:57:56 Re: Experimental patch for inter-page delay in VACUUM
Previous Message Neil Conway 2003-11-10 18:56:49 Re: Catching "UPDATE 0"