Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] New wal_sync_method for Darwin?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Chris Campbell <chris(at)bignerdranch(dot)com>
Cc: PostgreSQL-patches <pgsql-patches(at)postgresql(dot)org>
Subject: Re: [HACKERS] New wal_sync_method for Darwin?
Date: 2005-05-17 22:27:27
Message-ID: 200505172227.j4HMRRn27458@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-patches
Attached is an updated version of your Darwin Fsync-writethrough patch. 
I did much more restructuring because we do fsync in more places than
just WAL (like at checkpoints), and we should follow the WAL
write-through in those places too if that method is used for fsync of
WAL, rather than the normal fsync without the force-writethrough.

I restructured the code so now pg_fsync does writethrough or not based
on the wal setting.  My feeling is you need WAL writethrough, you need
it for other fsyncs as well.  Write-through was added in an 8.0.X
subrelease, and because it was a subrelease, the patch did a minimal
amount of code change.  This change is the restructuring that makes
sense for 8.1.

---------------------------------------------------------------------------

Chris Campbell wrote:
> Thanks, Bruce! Here's the patch (would you rather patches be attached 
> files, or inline?).
> 
> Some explanation. In the current code, there is no "fsync" on Windows, 
> only "fsync_writethrough". But, internally, it behaves identically to 
> "fsync". The only special case was for the GUC strings.
> 
> The patch changes it thusly: since "fsync" and "fsync_writethrough" 
> cannot be identical on Darwin, I added a new constant for 
> SYNC_METHOD_FSYNC_WRITE_THROUGH (3). A GUC value of 
> "fsync_writethrough" sets fsync_method to this new constant.
> 
> On Windows, the fsync_writethrough case falls through to the fsync case 
> (which simply runs fsync). On Darwin, the fsync_writethrough case 
> executes fcntl(openLogFile, F_FULLFSYNC, 0), which does everything that 
> fsync does, and then flushes the disk's write cache. The code does not 
> need to call fsync directly, just fcntl.
> 
> The performance of a script that executes 11,000 INSERT statements 
> (outside a transaction) is pretty slow, since it requires flushing the 
> disk write cache after EVERY query (transaction commit). I waited about 
> 15 minutes before I just killed it. But if you do all the inserts 
> inside a transaction, then I only pay the cache flush price once at 
> transaction commit, and it's quite usable.
> 
> I asked around on IRC, and the only place where flushing the disk cache 
> is important is when syncing a WAL file. So I didn't change the 
> behavior of fsync() globally for the code...just for WAL syncs.
> 
> I added a new #define called HAVE_FSYNC_WRITE_THROUGH, which is defined 
> only by the win32.h and darwin.h port headers. This controls the 
> availability of the "fsync_writethrough" GUC setting. Thanks to the 
> port headers, no configure test is needed.
> 
> Thanks! Let me know what you think of the patch, and tell me what else 
> I can do.
> 
> - Chris
> 
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

Responses

pgsql-patches by date

Next:From: Alvaro HerreraDate: 2005-05-17 23:16:12
Subject: Re: Refactoring in lock.c
Previous:From: Magnus HaganderDate: 2005-05-17 08:43:33
Subject: Re: Faarsi FAQ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group