Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS

From: Gasper Zejn <zejn(at)owca(dot)info>
To: Bruce Momjian <bruce(at)momjian(dot)us>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
Cc: Craig Ringer <craig(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Anthony Iliopoulos <ailiop(at)altatus(dot)com>, Andres Freund <andres(at)anarazel(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Catalin Iacob <iacobcatalin(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: PostgreSQL's handling of fsync() errors is unsafe and risks data loss at least on XFS
Date: 2018-04-04 17:23:58
Message-ID: 7fc5f714-1864-f928-851b-87fc4218c72a@owca.info
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 04. 04. 2018 15:49, Bruce Momjian wrote:
> I can understand why kernel developers don't want to keep failed sync
> buffers in memory, and once they are gone we lose reporting of their
> failure. Also, if the kernel is going to not retry the syncs, how long
> should it keep reporting the sync failure? To the first fsync that
> happens after the failure? How long should it continue to record the
> failure? What if no fsync() every happens, which is likely for
> non-Postgres workloads? I think once they decided to discard failed
> syncs and not retry them, the fsync behavior we are complaining about
> was almost required.
Ideally the kernel would keep its data for as little time as possible.
With fsync, it doesn't really know which process is interested in
knowing about a write error, it just assumes the caller will know how to
deal with it. Most unfortunate issue is there's no way to get
information about a write error.

Thinking aloud - couldn't/shouldn't a write error also be a file system
event reported by inotify? Admittedly that's only a thing on Linux, but
still.

Kind regards,
Gasper

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2018-04-04 17:25:11 Re: pgsql: Validate page level checksums in base backups
Previous Message Alvaro Herrera 2018-04-04 17:21:14 Re: Foreign keys and partitioned tables