Skip site navigation (1) Skip section navigation (2)

Re: Streaming replication, and walsender during recovery

From: Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>
To: Simon Riggs <simon(at)2ndQuadrant(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Streaming replication, and walsender during recovery
Date: 2010-01-28 19:00:33
Message-ID: 4B61DED1.7010409@enterprisedb.com (view raw or flat)
Thread:
Lists: pgsql-hackers
Simon Riggs wrote:
> On Thu, 2010-01-28 at 10:48 -0500, Tom Lane wrote:
>> Fujii Masao <masao(dot)fujii(at)gmail(dot)com> writes:
>>> How about just making a restore_command copy the WAL files as the
>>> normal one (e.g., 0000...) instead of a pg_xlog/RECOVERYXLOG?
>>> Though we need to worry about deleting them, we can easily leave
>>> the task to the bgwriter.
>> The reason for doing it that way was to limit disk space usage during
>> a long restore.  I'm not convinced we can leave the task to the bgwriter
>> --- it shouldn't be deleting anything at that point.
> 
> I think "bgwriter" means RemoveOldXlogFiles(), which would normally
> clear down files at checkpoint. If that was added to the end of
> RecoveryRestartPoint() to do roughly the same job then it could
> potentially work.

SR added a RemoveOldXLogFiles() call to CreateRestartPoint().

(Since 8.4, RecoveryRestartPoint() just writes the location of the
checkpoint record in shared memory, but doesn't actually perform the
restartpoint; bgwriter does that in CreateRestartPoint()).

> However, since not every checkpoint is a restartpoint we might easily
> end up with significantly more WAL files on the standby than would
> normally be there when it would be a primary. Not sure if that is an
> issue in this case, but we can't just assume we can store all files
> needed to restart the standby on the standby itself, in all cases. That
> might be an argument to add a restartpoint_segments parameter, so we can
> trigger restartpoints on WAL volume as well as time. But even that would
> not put an absolute limit on the number of WAL files.

I think it is a pretty important safety feature that we keep all the WAL
around that's needed to recover the standby. To avoid out-of-disk-space
situation, it's probably enough in practice to set checkpoint_timeout
small enough in the standby to trigger restartpoints often enough.

At the moment, we do retain streamed WAL as long as it's needed, but not
the WAL restored from archive.

-- 
  Heikki Linnakangas
  EnterpriseDB   http://www.enterprisedb.com

In response to

Responses

pgsql-hackers by date

Next:From: Simon RiggsDate: 2010-01-28 19:13:18
Subject: Re: Streaming replication, and walsender during recovery
Previous:From: Simon RiggsDate: 2010-01-28 18:58:19
Subject: Re: Streaming replication, and walsender during recovery

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group