Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: pgsql-hackers(at)postgresql(dot)org, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-performance(at)postgresql(dot)org, Michael Clemmons <glassresistor(at)gmail(dot)com>, Hannu Krosing <hannu(at)2ndquadrant(dot)com>, Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
Subject: Re: [HACKERS] Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Date: 2009-12-29 00:46:21
Message-ID: 4B39515D.6050309@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
Andres Freund wrote:
> As I said the real benefit only occurred after adding posix_fadvise(.., 
> FADV_DONTNEED) which is somewhat plausible, because i.e. the directory entries 
> don't need to get scheduled for every file and because the kernel can reorder a 
> whole directory nearly sequentially. Without the advice it the kernel doesn't 
> know in time that it should write that data back and it wont do it for 5 
> seconds by default on linux or such...
>   
I know they just fiddled with the logic in the last release, but for 
most of the Linux kernels out there now pdflush wakes up every 5 seconds 
by default.  But typically it only worries about writing things that 
have been in the queue for 30 seconds or more until you've filled quite 
a bit of memory, so that's also an interesting number.  I tried to 
document the main tunables here and describe how they fit together at 
http://www.westnet.com/~gsmith/content/linux-pdflush.htm

It would be interesting to graph the "Dirty" and "Writeback" figures in 
/proc/meminfo over time with and without this patch in place.  That 
should make it obvious what the kernel is doing differently in the two 
cases.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com  www.2ndQuadrant.com


In response to

Responses

pgsql-performance by date

Next:From: davidDate: 2009-12-29 00:46:26
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Previous:From: Andres FreundDate: 2009-12-29 00:43:15
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

pgsql-hackers by date

Next:From: davidDate: 2009-12-29 00:46:26
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)
Previous:From: Andres FreundDate: 2009-12-29 00:43:15
Subject: Re: Re: Faster CREATE DATABASE by delaying fsync (was 8.4.1 ubuntu karmic slow createdb)

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group