Skip site navigation (1) Skip section navigation (2)

Re: ext4 finally doing the right thing

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Jeff Davis <pgsql(at)j-davis(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: ext4 finally doing the right thing
Date: 2010-01-20 21:18:48
Message-ID: 4B577338.4090608@2ndquadrant.com (view raw or flat)
Thread:
Lists: pgsql-performance
Jeff Davis wrote:
>
>> On one side, we might finally be 
>> able to use regular drives with their caches turned on safely, taking 
>> advantage of the cache for other writes while doing the right thing with 
>> the database writes.
>>     
>
> That could be good news. What's your opinion on the practical
> performance impact? If it doesn't need to be fsync'd, the kernel
> probably shouldn't have written it to the disk yet anyway, right (I'm
> assuming here that the OS buffer cache is much larger than the disk
> write cache)?
>   

I know they just tweaked this area recently so this may be a bit out of 
date, but kernels starting with 2.6.22 allow you to get up to 10% of 
memory dirty before getting really aggressive about writing things out, 
with writes starting to go heavily at 5%.  So even with a 1GB server, 
you could easily find 100MB of data sitting in the kernel buffer cache 
ahead of a database write that needs to hit disc.  Once you start 
considering the case with modern hardware, where even my desktop has 8GB 
of RAM and most serious servers I see have 32GB, you can easily have 
gigabytes of such data queued in front of the write that now needs to 
hit the platter.

The dream is that a proper barrier implementation will then shuffle your 
important write to the front of that queue, without waiting for 
everything else to clear first.  The exact performance impact depends on 
how many non-database writes happen.  But even on a dedicated database 
disk, it should still help because there are plenty of non-sync'd writes 
coming out the background writer via its routine work and the checkpoint 
writes.  And the ability to fully utilize the write cache on the 
individual drives, on commodity hardware, without risking database 
corruption would make life a lot easier.

-- 
Greg Smith    2ndQuadrant   Baltimore, MD
PostgreSQL Training, Services and Support
greg(at)2ndQuadrant(dot)com  www.2ndQuadrant.com

In response to

Responses

pgsql-performance by date

Next:From: Carlo StonebanksDate: 2010-01-20 22:26:05
Subject: Re: Re: New server to improve performance on our large and busy DB - advice? (v2)
Previous:From: Kevin GrittnerDate: 2010-01-20 20:36:39
Subject: Re: New server to improve performance on our large and busy DB - advice?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group