Re: fdatasync performance problem with large number of DB files

On Thu, Mar 11, 2021 at 2:32 PM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> On Thu, Mar 11, 2021 at 2:25 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> > Trolling the net, I found a newer-looking version of the man page,
> > and behold it says
> >
> > In mainline kernel versions prior to 5.8, syncfs() will fail only
> > when passed a bad file descriptor (EBADF). Since Linux 5.8,
> > syncfs() will also report an error if one or more inodes failed
> > to be written back since the last syncfs() call.
> >
> > So this means that in less-than-bleeding-edge kernels, syncfs can
> > only be regarded as a dangerous toy. If we expose an option to use
> > it, there had better be large blinking warnings in the docs.
>
> Agreed. Perhaps we could also try to do something programmatic about that.

Time being of the essence, here is the patch I posted last year, this
time with a GUC and some docs. You can set sync_after_crash to
"fsync" (default) or "syncfs" if you have it.

I would plan to extend that to include a third option as already
discussed in the other thread, maybe something like "wal" (= sync WAL
files and then do extra analysis of WAL data to sync only data
modified since checkpoint but not replayed), but that'd be material
for PG15.