Skip site navigation (1) Skip section navigation (2)

Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-committers(at)postgresql(dot)org
Cc: Greg Stark <stark(at)postgresql(dot)org>
Subject: Re: pgsql: Speed up CREATE DATABASE by deferring the fsyncs until after
Date: 2010-02-21 23:43:26
Message-ID: 201002220043.29420.andres@anarazel.de (view raw or flat)
Thread:
Lists: pgsql-committerspgsql-hackers
On Monday 15 February 2010 01:50:57 Greg Stark wrote:
> Log Message:
> -----------
> Speed up CREATE DATABASE by deferring the fsyncs until after copying
> all the data and using posix_fadvise to nudge the OS into flushing it
> earlier. This also hopefully makes CREATE DATABASE avoid spamming the
> cache.
> 
> Tests show a big speedup on Linux at least on some filesystems.
> 
> Idea and patch from Andres Freund.
I just found a relatively big problem with one of your modifications on the 
patch - you removed the 
FreeDir(xldir);
xldir = AllocateDir(fromdir);
pair - unfortunately its crucial because otherwise the DIR does not get 
rewound - that resulted in *no* files getting fsync()ed (otherwise the loop 
above wouldn't have finished yet...).
I think that was also causing the problems I pointed out in " Directory fsync 
and other fun"...

You removed it because you didn't want to open the directory twice? I think 
doing that is simpler than using rewinddir - I have no idea how usable that 
one is on windows for example

Could you add it back?

Andres

In response to

Responses

pgsql-hackers by date

Next:From: Greg StarkDate: 2010-02-22 00:11:05
Subject: pgsql: Oops, don't forget to rewind the directory before scanning it to
Previous:From: Tom LaneDate: 2010-02-21 22:16:54
Subject: Re: PGXS: REGRESS_OPTS=--load-language=plpgsql

pgsql-committers by date

Next:From: Greg StarkDate: 2010-02-22 00:11:05
Subject: pgsql: Oops, don't forget to rewind the directory before scanning it to
Previous:From: Andrew DunstanDate: 2010-02-20 22:24:29
Subject: pgsql: Adjust sample auto-explain output to reflect query text

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group