Re: [HACKERS] New wal_sync_method for Darwin?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Chris Campbell <chris(at)bignerdranch(dot)com>
Cc: PostgreSQL-patches <pgsql-patches(at)postgresql(dot)org>
Subject: Re: [HACKERS] New wal_sync_method for Darwin?
Date: 2005-05-17 22:27:27
Message-ID: 200505172227.j4HMRRn27458@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches


Attached is an updated version of your Darwin Fsync-writethrough patch.
I did much more restructuring because we do fsync in more places than
just WAL (like at checkpoints), and we should follow the WAL
write-through in those places too if that method is used for fsync of
WAL, rather than the normal fsync without the force-writethrough.

I restructured the code so now pg_fsync does writethrough or not based
on the wal setting. My feeling is you need WAL writethrough, you need
it for other fsyncs as well. Write-through was added in an 8.0.X
subrelease, and because it was a subrelease, the patch did a minimal
amount of code change. This change is the restructuring that makes
sense for 8.1.

---------------------------------------------------------------------------

Chris Campbell wrote:
> Thanks, Bruce! Here's the patch (would you rather patches be attached
> files, or inline?).
>
> Some explanation. In the current code, there is no "fsync" on Windows,
> only "fsync_writethrough". But, internally, it behaves identically to
> "fsync". The only special case was for the GUC strings.
>
> The patch changes it thusly: since "fsync" and "fsync_writethrough"
> cannot be identical on Darwin, I added a new constant for
> SYNC_METHOD_FSYNC_WRITE_THROUGH (3). A GUC value of
> "fsync_writethrough" sets fsync_method to this new constant.
>
> On Windows, the fsync_writethrough case falls through to the fsync case
> (which simply runs fsync). On Darwin, the fsync_writethrough case
> executes fcntl(openLogFile, F_FULLFSYNC, 0), which does everything that
> fsync does, and then flushes the disk's write cache. The code does not
> need to call fsync directly, just fcntl.
>
> The performance of a script that executes 11,000 INSERT statements
> (outside a transaction) is pretty slow, since it requires flushing the
> disk write cache after EVERY query (transaction commit). I waited about
> 15 minutes before I just killed it. But if you do all the inserts
> inside a transaction, then I only pay the cache flush price once at
> transaction commit, and it's quite usable.
>
> I asked around on IRC, and the only place where flushing the disk cache
> is important is when syncing a WAL file. So I didn't change the
> behavior of fsync() globally for the code...just for WAL syncs.
>
> I added a new #define called HAVE_FSYNC_WRITE_THROUGH, which is defined
> only by the win32.h and darwin.h port headers. This controls the
> availability of the "fsync_writethrough" GUC setting. Thanks to the
> port headers, no configure test is needed.
>
> Thanks! Let me know what you think of the patch, and tell me what else
> I can do.
>
> - Chris
>
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

Attachment Content-Type Size
unknown_filename text/plain 10.0 KB

Responses

Browse pgsql-patches by date

  From Date Subject
Next Message Alvaro Herrera 2005-05-17 23:16:12 Re: Refactoring in lock.c
Previous Message Magnus Hagander 2005-05-17 08:43:33 Re: Faarsi FAQ?