Re: initdb and fsync

On Fri, Jun 22, 2012 at 10:04:23AM -0400, Robert Haas wrote:
> On Sat, Feb 4, 2012 at 8:18 PM, Noah Misch <noah(at)leadboat(dot)com> wrote:
> > If we add fsync calls to the initdb process, they should cover the entire data
> > directory tree. ?This patch syncs files that initdb.c writes, but we ought to
> > also sync files that bootstrap-mode backends had written. ?An optimization
> > like the pg_flush_data() call in copy_file() may reduce the speed penalty.
> >
> > initdb should do these syncs by default and offer an option to disable them.
>
> This may be a stupid question, by why is it initdb's job to fsync the
> files the server creates, rather than the server's job? Normally we
> rely on the server to make its own writes persistent.

Modularity would dictate having the server fsync its own work product, but I
expect that approach to perform materially worse. initdb runs many
single-user server instances, and each would fsync independently. When N
initdb steps change one file, it would see N fsyncs. Using sync_file_range to
queue all writes is best for the typical interactive or quasi-interactive
initdb user. It's not always a win for server fsyncs, so we would need to
either define special cases such that it's used during initdb or forgo the
optimization. On the other hand, the server could skip some files, like fsm
forks, in a principled manner.

Overall, I think it will hard to improve modularity while retaining the
performance Jeff's approach achieves through exploiting initdb's big-picture
perspective. So I favor how Jeff has implemented it.

nm