RE: Changing default value of wal_sync_method to open_datasync on Linux

From: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
To: 'Andres Freund' <andres(at)anarazel(dot)de>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Changing default value of wal_sync_method to open_datasync on Linux
Date: 2018-02-20 01:09:37
Message-ID: 0A3221C70F24FB45833433255569204D1F8CDF77@G01JPEXMBYT05
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Andres Freund [mailto:andres(at)anarazel(dot)de]
> Indeed. My past experience with open_datasync on linux shows it to be slower
> by roughly an order of magnitude. Even if that would turn out not to be
> the case anymore, I'm *extremely* hesitant to make such a change.

Thanks for giving so quick feedback. An order of magnitude is surprising. Can you share the environment (Linux distro version, kernel version, filesystem, mount options, workload, etc.)? Do you think of anything that explains the degradation? I think it is reasonable that open_datasync is faster than fdatasync because:

* Short transactions like pgbench require less system calls: write()+fdatasync() vs write().
* fdatasync() probably has to scan the page cache for dirty pages.

The above differences should be invisible on slow disks, but they will show up on faster storage. I guess that's why Robert said open_datasync was much faster on NVRAM.

The manual says that pg_test_fsync is a tool for selecting wal_sync_method value, and it indicates open_datasync is better. Why is fdatasync the default value only on Linux? I don't understand as a user why PostgreSQL does the special handling. If the current behavior of choosing fdatasync by default is due to some deficiency of old kernel and/or filesystem, I think we can change the default so that most users don't have to change wal_sync_method.

Regards
Takayuki Tsunakawa

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2018-02-20 01:21:12 Re: master check fails on Windows Server 2008
Previous Message Mark Kirkwood 2018-02-20 00:57:54 Re: Changing default value of wal_sync_method to open_datasync on Linux