Re: pg_dump, pg_dumpall and data durability

On Mon, Nov 7, 2016 at 7:52 PM, Michael Paquier
<michael(dot)paquier(at)gmail(dot)com> wrote:
> On Tue, Nov 8, 2016 at 1:24 AM, Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at> wrote:
>> The patch does not apply, I had to change the hunk for
>> src/include/common/file_utils.h.
>
> Yes, the patch has rotten a bit because of f82ec32a. 5d58c07a has also
> made the --noxxx option names appearing as --no-xxx.
>
>> Also, compilation fails because function "pre_fsync_fname" cannot be
>> resolved during linking. Is that a typo for "pre_fsync_fname" or is
>> the patch incomplete?
>
> A typo s/pre_fsync_fname/pre_sync_fname, and a mistake from me because
> I did not compile with -DPG_FLUSH_DATA_WORKS to check this code.
>
> v2 is attached, fixing those issues.

Kyotaro Horiguchi set this patch to "Ready for Committer" in the
CommitFest application, but that seems entirely premature to me
considering that v2 has had no review at all, or at least not that I
see on this thread. I'm setting it back to "Needs Review".

First question: Do we even want this? Generally, when a program
writes to a file, we rely on the operating system to decide when that
data should be written back to disk. We have to override that
distinction for things internal to PostgreSQL because we need certain
bits of data to reach the disk in a certain order, but it's unclear to
me how far outside the core database system we want to extend that.
Are we going to have psql fsync() data it writes to a file when \o is
used, for example? That would seem to me to be beyond insane, because
we have no idea whether the user actually needs that file to be
durable. It is a better bet that a pg_dump command's output needs
durability, of course, but I feel that we shouldn't just go disabling
the filesystem cache one program at a time without some guiding
principle.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company