Skip site navigation (1) Skip section navigation (2)

Re: Load distributed checkpoint

From: "Inaam Rana" <inaamrana(at)gmail(dot)com>
To: "Takayuki Tsunakawa" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Load distributed checkpoint
Date: 2006-12-20 20:29:30
Message-ID: 833c669b0612201229o2452e4efp5967a245d1e10692@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
On 12/20/06, Takayuki Tsunakawa <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:
>
> [Conclusion]
> I believe that the problem cannot be solved in a real sense by
> avoiding fsync/fdatasync().  We can't ignore what commercial databases
> have done so far.  The kernel does as much as he likes when PostgreSQL
> requests him to fsync().
>

I am new to the community and am very interested in the tests that you have
done. I am also working on resolving the sudden IO spikes at checkpoint
time. I agree with you that fsync() is the core issue here.

Being a new member I was wondering if someone on this list has done testing
with O_DIRECT and/or O_SYNC for datafiles as that seems to be the most
logical way of dealing with fsync() flood at checkpoint time. If so, I'll be
very interested in the results. As mentioned in this thread that a single
bgwriter with O_DIRECT will not be able to keep pace with cleaning effort
causing backend writes. I think (i.e. IMHO) multiple bgwriters and/or
AsyncIO with O_DIRECT can resolve this issue.

Talking of bgwriter_* parameters I think we are missing a crucial internal
counter i.e. number of dirty pages. How much work bgwriter has to do at each
wakeup call should be a function of total buffers and currently dirty
buffers. Relying on both these values instead of just one static NBuffers
should allow bgwriter to adapt more quickly to workload changes and ensure
that not much work is accumulated for checkpoint.

-- 
Inaam Rana
EnterpriseDB   http://www.enterprisedb.com

In response to

Responses

pgsql-hackers by date

Next:From: Zdenek KotalaDate: 2006-12-20 21:05:50
Subject: Re: Question about debugging bootstrapping and catalog
Previous:From: Lukas Kahwe SmithDate: 2006-12-20 20:26:12
Subject: Re: Release 8.2.0 done, 8.3 development starts

pgsql-patches by date

Next:From: Russell SmithDate: 2006-12-20 21:23:49
Subject: Re: column ordering, was Re: [PATCHES] Enums patch v2
Previous:From: Martijn van OosterhoutDate: 2006-12-20 20:17:12
Subject: Re: column ordering, was Re: [PATCHES] Enums patch v2

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group