Re: fdatasync(2) on macOS

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: fdatasync(2) on macOS
Date: 2021-01-18 03:39:58
Message-ID: CA+hUKGLv-kvrtA5QEL=p=dYK0p9gsMXJaVhUFu+A-KyFrFi=2g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sat, Jan 16, 2021 at 6:55 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> So it's not a no-op, but on the other hand it's not succeeding in getting
> bits down to the platter. I'm not inclined to dike it out, but it does
> seem problematic that we're defaulting to open_datasync, which is also
> not getting bits down to the platter.

Hmm, OK, from these times it does appear that O_SYNC and O_DSYNC
actually do something then. It's baffling that they are undocumented.
It might be possible to use dtrace on a SIP-disabled Mac to trace the
IOs with this script, to see if the B_FUA flag is being set, which
might make open_datasync better than fdatasync (if it's being sent and
not ignored), but again, who knows?!:

https://github.com/apple/darwin-xnu/blob/master/bsd/dev/dtrace/scripts/io.d

> I have a vague recollection that we discussed changing the default
> wal_sync_method for Darwin years ago, but I don't recall why we
> didn't pull the trigger. These results certainly suggest that
> we oughta.

No strong preference here, at least without more information. It's
unsettling that two of our wal_sync_methods are based on half-released
phantom operating system features, but there doesn't seem to be much
we can do about that other than try to understand what they do. I see
that the idea of defaulting to fsync_writethrough was discussed a
decade ago and rejected[1]. I'm not entirely sure how it manages to
be so slow.

It looks like the reliability section of our manual could use a spring
clean[2]. It's still talking about IDE and platters, instead of
modern stuff like NVMe, cloud/network storage and FUA flags.

[1] https://www.postgresql.org/message-id/flat/AANLkTik261QWc9kGv6acZz2h9ZrQy9rKQC8ow5U1tAaM%40mail.gmail.com
[2] https://www.postgresql.org/docs/13/wal-reliability.html

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Etsuro Fujita 2021-01-18 04:06:23 Re: Asynchronous Append on postgres_fdw nodes.
Previous Message Bharath Rupireddy 2021-01-18 03:33:56 Re: [PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit