Re: initdb and fsync

On Thursday, March 15, 2012 07:38:38 AM Jeff Davis wrote:
> On Wed, 2012-03-14 at 10:26 +0100, Andres Freund wrote:
> > On Wednesday, March 14, 2012 05:23:03 AM Jeff Davis wrote:
> > > On Tue, 2012-03-13 at 09:42 +0100, Andres Freund wrote:
> > > > for recursively everything in dir:
> > > > posix_fadvise(fd, POSIX_FADV_DONTNEED);
> > > >
> > > > for recursively everything in dir:
> > > > fsync(fd);
> > >
> > > Wow, that made a huge difference!
> > >
> > > no sync: ~ 1.0s
> > > sync: ~10.0s
> > > fadvise+sync: ~ 1.3s
>
> I take that back. There was something wrong with my test -- fadvise
> helps, but it only takes it from ~10s to ~6.5s. Not quite as good as I
> hoped.
Thats surprising. I wouldn't expect such a big difference between fadvise +
sync_file_range. Rather strange.

> > Well, while the positive effect of this are rather large it also has the
> > bad effect of pushing the whole new database out of the cache. Which is
> > not so nice if you want to run tests on it seconds later.
>
> I was unable to see a regression when it comes to starting it up after
> the fadvise+fsync. My test just started the server, created a table,
> then stopped the server. It was actually a hair faster with the
> directory that had been fadvise'd and then fsync'd, but I assume that
> was noise. Regardless, this doesn't look like an issue.
Hm. Ok.

> > How are the results with sync_file_range(fd, 0, 0,
> > SYNC_FILE_RANGE_WRITE)?
> That is much faster than using fadvise. It goes down to ~2s.

> Unfortunately, that's non-portable. Any other ideas? 6.5s a little on
> the annoying side (and causes some disconcerting sounds to come from my
> disk), especially when we _know_ it can be done in 2s.
Its not like posix_fadvise is actually portable. So I personally don't see a
problem with that, but...

Andres