Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] win32 performance - fsync question

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Magnus Hagander <mha(at)sollentuna(dot)net>
Cc: Michael Paesold <mpaesold(at)gmx(dot)at>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>,pgsql-hackers-win32(at)postgresql(dot)org,Merlin Moncure <merlin(dot)moncure(at)rcsonline(dot)com>,PostgreSQL-patches <pgsql-patches(at)postgresql(dot)org>
Subject: Re: [HACKERS] win32 performance - fsync question
Date: 2005-03-24 04:31:15
Message-ID: 200503240431.j2O4VFG07349@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-hackers-win32pgsql-patches
I have applied the following patch to CVS HEAD and 8.0.X that changes
the Win32 O_SYNC flag to O_DATASYNC, because this the actual behavior of
the flag.  This is now the default wal fsync method on Win32 because we
perfer O_DATASYNC to fsync().

And second, it changes Win32 fsync to a new wal sync method called
fsync_writethrough which is the old Win32 fsync behavior, which uses
_commit().

---------------------------------------------------------------------------

Magnus Hagander wrote:
> > > > > * Win32, with fsync, write-cache disabled: no data corruption
> > > > > * Win32, with fsync, write-cache enabled: no data corruption
> > > > > * Win32, with osync, write cache disabled: no data corruption
> > > > > * Win32, with osync, write cache enabled: no data 
> > corruption. Once 
> > > > > I
> > > > > got:
> > > > > 2005-02-24 12:19:54 LOG:  could not open file "C:/Program 
> > > > > Files/PostgreSQL/8.0/data/pg_xlog/000000010000000000000010"
> > > > (log file
> > > > > 0, segment 16): No such file or directory
> > > > >   but the data in the database was consistent.
> > > > 
> > > > It disturbs me that you couldn't produce data corruption in the 
> > > > cases where it theoretically should occur.  Seems like this is an 
> > > > indication that your test was insufficiently severe, or 
> > that there 
> > > > is something going on we don't understand.
> > > 
> > > The Windows driver knows abotu the write cache, and at 
> > least fsync() 
> > > pushes through the write cache even if it's there. This seems to 
> > > indicate taht O_SYNC at least partiallyi does this as well. This is 
> > > why there is no performance difference at all on fsync() with write 
> > > cache on or off.
> > > 
> > > I don't know if this is true for all IDE disks. COuld be 
> > that my disk 
> > > is particularly well-behaved.
> > 
> > This indicated to me that open_sync did not require any 
> > additional changes than our current fsync.
> 
> fsync and open_sync both write through the write cache in the operating
> system. Only fsync=off turns this off.
> 
> fsync also writes through the hardware write cache. o_sync does not.
> This is what causes the large slowdown with write cache enabled,
> *including* most battery backed write cache systems (pretty much making
> the write-cache a waste of money). This may be a good thing on IDE
> systems (for admins that don't know how to remove the little check in
> the box for "enable write caching on the disk" that MS provides, which
> *explicitly* warns that you may lose data if you enabled it), but it's a
> very bad thing for anything higher end.
> 
> fsync also syncs the directory metadata. o_sync only cares about the
> files contents. (This is what causes the large slowdown with write cache
> *disabled*, becuase it requires multiple writes on multiple disk
> locations for each fsync). 
> 
> 
> Basically, fsync hurts people who configure their box correctly, or who
> use things like SCSI disks. o_sync hurts people who configure their
> machine in an unsafe way.
> 
> //Magnus
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

In response to

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2005-03-24 05:15:53
Subject: Re: fool-toleranced optimizer
Previous:From: Christopher Kings-LynneDate: 2005-03-24 01:03:04
Subject: Re: \x in psql

pgsql-patches by date

Next:From: Bruce MomjianDate: 2005-03-24 04:42:56
Subject: Link to pgport earlier
Previous:From: Michael FuhrDate: 2005-03-24 04:23:43
Subject: Re: PL/Python patch for Universal Newline Support

pgsql-hackers-win32 by date

Next:From: Andrew SullivanDate: 2005-03-24 13:17:49
Subject: Re: Simple query takes a long time on win2K
Previous:From: A. MousDate: 2005-03-24 03:49:22
Subject: Re: Simple query takes a long time on win2K

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group