Re: Directory fsync and other fun

From: Takahiro Itagaki <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: pgsql-hackers(at)postgresql(dot)org, Mark Mielke <mark(at)mark(dot)mielke(dot)cc>, Florian Weimer <fw(at)deneb(dot)enyo(dot)de>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Greg Stark <gsstark(at)mit(dot)edu>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Directory fsync and other fun
Date: 2010-02-24 02:51:07
Message-ID: 20100224115107.9738.52131E4D@oss.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


Andres Freund <andres(at)anarazel(dot)de> wrote:

> I started setting up some halfway automated method of simulating hard crashes
> and even while setting those up I found some pretty unsettling results...
> Now its not unlikely that my testing is flawed but unfortunately I don't see
> where right now (its 3am now and I have a 8h trainride behind me, so ...)

I think the reported behavior is a TODO item to research:

* Research use of fsync to a newly created directory.
There is no guarantee that mkdir() flushes metadata of the directory.

Also, I heard ext4 has a "feature" in that rename() might truncate the
renamed file to zero bytes on crash. The user data in the file might be
lost if the machine crashes just after rename().

* Research whether our use of rename() is safe on various file systems.
Especially on ext4, the contents of the renamed file might be lost on crash.

Comments and suggestion for improvements of words welcome.
If no objection, I'll add them at the Fsync section in the TODO page.

Regards,
---
Takahiro Itagaki
NTT Open Source Software Center

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2010-02-24 02:59:49 Re: Directory fsync and other fun
Previous Message Tom Lane 2010-02-24 02:46:14 Re: [PATCH] 8.5 TODO: Add comments to output indicating version of pg_dump and of the database server