Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

From: Andres Freund <andres(at)anarazel(dot)de>
To: Michael Clemmons <glassresistor(at)gmail(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org, Greg Smith <greg(at)2ndquadrant(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-performance(at)postgresql(dot)org, Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Date: 2009-12-29 03:11:14
Message-ID: 200912290411.15659.andres@anarazel.de (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
On Tuesday 29 December 2009 04:04:06 Michael Clemmons wrote:
> Maybe not crash out but in this situation.
> N=0
> while(N>=0):
>     CREATE DATABASE new_db_N;
> Since the fsync is the part which takes the memory and time but is
>  happening in the background want the fsyncs pile up in the background
>  faster than can be run filling up the memory and stack.
> This is very likely a mistake on my part about how postgres/processes
The difference should not be visible outside the "CREATE DATABASE ..." at all.
Currently the process simplifiedly works like:

------------
for file in source directory:
	copy_file(source/file, target/file);
	fsync(target/file);
------------

I changed it to:

-------------
for file in source directory:
	copy_file(source/file, target/file);

	/*please dear kernel, write this out, but dont block*/
	posix_fadvise(target/file, FADV_DONTNEED); 

for file in source directory:
	fsync(target/file);
-------------

If at any point in time there is not enough cache available to cache anything 
copy_file() will just have to wait for the kernel to write out the data.
fsync() does not use memory itself.

Andres

In response to

pgsql-performance by date

Next:From: Greg StarkDate: 2009-12-29 10:48:10
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Previous:From: Michael ClemmonsDate: 2009-12-29 03:04:06
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2009-12-29 03:32:03
Subject: Re: Serializable implementation
Previous:From: Michael ClemmonsDate: 2009-12-29 03:04:06
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group