Skip site navigation (1) Skip section navigation (2)

Re: Experimental patch for inter-page delay in VACUUM

From: Jan Wieck <JanWieck(at)Yahoo(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Ang Chin Han <angch(at)bytecraft(dot)com(dot)my>,Christopher Browne <cbbrowne(at)acm(dot)org>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Experimental patch for inter-page delay in VACUUM
Date: 2003-11-04 15:45:22
Message-ID: 3FA7C992.5090106@Yahoo.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Tom Lane wrote:

> Jan Wieck <JanWieck(at)Yahoo(dot)com> writes:
>> What still needs to be addressed is the IO storm cause by checkpoints. I 
>> see it much relaxed when stretching out the BufferSync() over most of 
>> the time until the next one should occur. But the kernel sync at it's 
>> end still pushes the system hard against the wall.
> 
> I have never been happy with the fact that we use sync(2) at all.  Quite
> aside from the "I/O storm" issue, sync() is really an unsafe way to do a
> checkpoint, because there is no way to be certain when it is done.  And
> on top of that, it does too much, because it forces syncing of files
> unrelated to Postgres.

Sure does it do too much. But together with the other layer of 
indirection, the virtual file descriptor pool, what is the exact 
guaranteed behaviour of

     write(); close(); open(); fsync();

cross platform?


> Actually, once you build it this way, you could make all writes
> synchronous (open the files O_SYNC) so that there is never any need for
> explicit fsync at checkpoint time.  The background writer process would
> be the one incurring the wait in most cases, and that's just fine.  In
> this way you could directly control the rate at which writes are issued,
> and there's no I/O storm at all.  (fsync could still cause an I/O storm
> if there's lots of pending writes in a single file.)

Yes, but then the configuration leans more towards "take over the RAM" 
again, and we better have a much improved cache strategy before that.


Jan

-- 
#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me.                                  #
#================================================== JanWieck(at)Yahoo(dot)com #


In response to

Responses

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2003-11-04 15:51:02
Subject: Re: Experimental patch for inter-page delay in VACUUM
Previous:From: Tom LaneDate: 2003-11-04 15:31:39
Subject: Re: Experimental patch for inter-page delay in VACUUM

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group