Skip site navigation (1) Skip section navigation (2)

Re: Incremental checkopints

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Greg Smith <greg(at)2ndquadrant(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Incremental checkopints
Date: 2011-08-03 16:10:32
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-hackers
2011/7/29 Greg Smith <greg(at)2ndquadrant(dot)com>:
> 1) Postponing writes as long as possible always improves the resulting
> throughput of those writes.  Any incremental checkpoint approach will detune
> throughput by some amount.  If you make writes go out more often, they will
> be less efficient; that's just how things work if you benchmark anything
> that allows write combining.  Any incremental checkpoint approach is likely
> to improve latency in some cases if it works well, while decreasing
> throughput in most cases.

Agreed.  I came to the same conclusion a while back and then got
depressed.  That might mean we need a parameter to control the
behavior, unless we can find a change where the throughput drop is
sufficiently small that we don't really care, or make the optimization
apply only in cases where we determine that the latency problem will
be so severe that we'll certainly be willing to accept a drop in
throughput to avoid it.

> 2) The incremental checkpoint approach used by other databases, such as the
> MySQL implementation, works by tracking what transaction IDs were associated
> with a buffer update.  The current way PostgreSQL saves buffer sync
> information for the checkpoint to process things doesn't store enough
> information to do that.  As you say, the main price there is some additional
> memory.

I think what we'd need to track is the LSN that first dirtied the page
(as opposed to the current field, which tracks the LSN that most
recently wrote the page).  If we write and flush all pages whose
first-dirtied LSN precedes some cutoff point, then we ought to be able
to advance the redo pointer to that point.

Robert Haas
The Enterprise PostgreSQL Company

In response to

pgsql-hackers by date

Next:From: Robert HaasDate: 2011-08-03 16:14:15
Subject: Re: FOR KEY LOCK foreign keys
Previous:From: Robert HaasDate: 2011-08-03 15:57:12
Subject: Re: WIP fix proposal for bug #6123

Privacy Policy | About PostgreSQL
Copyright © 1996-2018 The PostgreSQL Global Development Group