Skip site navigation (1) Skip section navigation (2)

Re: Load distributed checkpoint

From: Greg Smith <gsmith(at)gregsmith(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Load distributed checkpoint
Date: 2006-12-22 21:33:43
Message-ID: Pine.GSO.4.64.0612221612050.24353@westnet.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
On Fri, 22 Dec 2006, Simon Riggs wrote:

> I have also seen cases where the WAL drive, even when separated, appears
> to spike upwards during a checkpoint. My best current theory, so far
> untested, is that the WAL and data drives are using the same CFQ
> scheduler and that the scheduler actively slows down WAL requests when
> it need not. Mounting the drives as separate block drives with separate
> schedulers, CFQ for data and Deadline for WAL should help.

The situation I've been seeing is that the database needs a new block to 
complete a query and issues a read request to get it, but that read is 
behind the big checkpoint fsync.  Client sits there for quite some time 
waiting for the fsync to finish before it gets the data it needs, and now 
your trivial select took seconds to complete.  It's fairly easy to 
replicate this problem using pgbench on Linux--I've seen a query sit there 
for 15 seconds when going out of my way to aggrevate the behavior.  One of 
Takayuki's posts here mentioned a worst-case delay of 13 seconds, that's 
the problem rearing its ugly head.

You may be right that what you're seeing would be solved with a more 
complicated tuning on a per-device basis (which, by the way, isn't 
available unless you're running a more recent Linux kernel than most many 
distributions have available).  You can tune the schedulers all day and 
not make a lick of difference to what I've been running into; I know, I 
tried.

--
* Greg Smith gsmith(at)gregsmith(dot)com http://www.gregsmith.com Baltimore, MD

In response to

pgsql-hackers by date

Next:From: Gregory StarkDate: 2006-12-22 21:41:35
Subject: Re: Companies Contributing to Open Source
Previous:From: Gregory StarkDate: 2006-12-22 21:21:25
Subject: Re: Operator class group proposal

pgsql-patches by date

Next:From: Gregory StarkDate: 2006-12-22 21:48:37
Subject: Re: WIP patch for "operator families"
Previous:From: Gregory StarkDate: 2006-12-22 21:13:49
Subject: Re: Load distributed checkpoint

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group