Re: [HACKERS] WAL logging freezing

From: "Simon Riggs" <simon(at)2ndquadrant(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, "Heikki Linnakangas" <heikki(at)enterprisedb(dot)com>, <pgsql-hackers(at)postgresql(dot)org>, <pgsql-patches(at)postgresql(dot)org>
Subject: Re: [HACKERS] WAL logging freezing
Date: 2006-10-30 20:30:19
Message-ID: 1162240219.11568.320.camel@silverbirch.site
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

On Mon, 2006-10-30 at 12:05 -0500, Tom Lane wrote:
> Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> > Ugh. Is there another solution to this? Say, sync the buffer so that
> > the hint bits are written to disk?
>
> Yeah. The original design for all this is explained by the notes for
> TruncateCLOG:
>
> * When this is called, we know that the database logically contains no
> * reference to transaction IDs older than oldestXact. However, we must
> * not truncate the CLOG until we have performed a checkpoint, to ensure
> * that no such references remain on disk either; else a crash just after
> * the truncation might leave us with a problem.
>
> The pre-8.2 coding is actually perfectly safe within a single database,
> because TruncateCLOG is only called at the end of a database-wide
> vacuum, and so the checkpoint is guaranteed to have flushed valid hint
> bits for all tuples to disk. There is a risk in other databases though.
> I think that in the 8.2 structure the equivalent notion must be that
> VACUUM has to flush and fsync a table before it can advance the table's
> relminxid.

Ouch! We did discuss that also. Flushing the buffercache is nasty with
very large caches, so this makes autovacuum much less friendly - and
could take a seriously long time if you enforce the vacuum delay
costings.

ISTM we only need to flush iff the clog would be truncated when we
update relminxid. Otherwise we are safe to update even if we crash,
since the clog will not have been truncated.

> That still leaves us with the problem of hint bits not being updated
> during WAL replay. I think the best solution for this is for WAL replay
> to force relvacuumxid to equal relminxid (btw, these field names seem
> poorly chosen, and the comment in catalogs.sgml isn't self-explanatory...)
> rather than adopting the value shown in the WAL record. This probably
> is best done by abandoning the generic "overwrite tuple" WAL record type
> in favor of something specific to minxid updates. The effect would then
> be that a PITR slave would not truncate its clog beyond the freeze
> horizon until it had performed a vacuum of its own.

Sounds good. Methinks we do still need the TruncateCLOG patch to ensure
we do WAL replay for the truncation? I'm posting that now to -patches as
a prototype.

--
Simon Riggs
EnterpriseDB http://www.enterprisedb.com

Attachment Content-Type Size
xlog_clog_truncate.patch text/x-patch 3.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Simon Riggs 2006-10-30 20:49:57 Re: Deadlock with pg_dump?
Previous Message Chris Browne 2006-10-30 17:23:18 Re: [HACKERS] Replication documentation addition

Browse pgsql-patches by date

  From Date Subject
Next Message Simon Riggs 2006-10-30 20:49:57 Re: Deadlock with pg_dump?
Previous Message Tom Lane 2006-10-30 17:05:19 Re: WAL logging freezing