Skip site navigation (1) Skip section navigation (2)

Infrastructure changes for recovery (v8)

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: List pgsql-patches <pgsql-patches(at)postgresql(dot)org>
Subject: Infrastructure changes for recovery (v8)
Date: 2008-09-30 22:52:31
Message-ID: 1222815151.4445.1397.camel@ebony.2ndQuadrant (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patches
Patch now includes all previous agreed changes, plus I've found what
looks to be a workable method of removing the shutdown checkpoint
without loss of robustness. 

Patch summary

Tuning 
* Bgwriter performs dirty block cleaning during recovery
* Bgwriter performs restartpoints, offloading this task from Startup
process to allow it to continue with recovery actions 
* Shutdown checkpoint removed at end of recovery. Bgwriter performs
immediate checkpoint instead, so we have same protection, but
connections and transactions can be started earlier than previously.
* PreAllocXLogs() not performed by startup process, so we do not delay
startup while we write zeroes to next WAL file. bgwriter does that now.
* XLogCtl structure padding for enhanced scalability

Recovery State Changes
* If archive recovery proceeds past a safe stopping point we signal the
postmaster that database is now in a consistent state, PM_RECOVERY. This
state change is also linked to startup of the bgwriter and stats
processes (and will in the future be the place where read only backends
may connect also)
* optional recovery_safe_start_location parameter now provided in
recovery.conf, to allow a consistency point to be manually defined if a
base backup was not taken using standard pg_start/stop backup functions
* New minSafeStopPoint added to controlfile to allow us to determine
consistency if archive recovery crashes/restarts. Value is updated each
time we access new WAL file.
* stats file removed earlier in recovery, so we may accumulate new stats
during recovery
* End of recovery is now marked by a clear global state change. Change
is global, atomic and fast - tested for using IsRecoveryProcessingMode()

Additional Safeguards
* Locks are placed around all ControlFile operations
* XLogInsert() and AssignTransactionId() now have specific checks to
prevent their use during recovery
* Makes StartupMultiXact() atomic. Adds comments to show that
StartCLOG() is already atomic, though StartupSUBTRANS() is not (this
will be addressed in a later patch, so not touched here)
* recovery.conf is not removed until slightly later now, to protect
against crash at the end of startup
* New WAL record XLOG_RECOVERY_END is now only place where timelineid
may change

Other Changes
* log_restartpoints removed, use log_checkpoints in postgresql.conf
* pg_controldata and pg_resetxlog changed to show safe start point
* designed to work in EXEC_BACKEND mode for Windows
* additional function signature for pg_start_backup('label', true |
false) to allow definition of immediate checkpoint/not
* doc changes for recovery.conf parameters
* fixes bug discovered while other testing: if pg_stop_backup() is run
when xlogswitch has just occurred then we do not switch log files, yet
we return current filename even though nothing of value in it. If
archive_timeout not enabled we would wait forever for pg_stop_backup()
to return. 
* Substantial comments throughout

Patch is now v8.

 doc/src/sgml/backup.sgml                 |   30 !
 doc/src/sgml/func.sgml                   |   12 
 src/backend/access/transam/clog.c        |    3 
 src/backend/access/transam/multixact.c   |   14 
 src/backend/access/transam/subtrans.c    |    3 
 src/backend/access/transam/xact.c        |    3 
 src/backend/access/transam/xlog.c        |  783 ++++++++++++++-!!!!!!!!!!!!!!!
 src/backend/postmaster/bgwriter.c        |  418 +++--!!!!!!!!!
 src/backend/postmaster/postmaster.c      |   62 +!
 src/backend/storage/buffer/README        |    9 
 src/bin/pg_controldata/pg_controldata.c  |    3 
 src/bin/pg_resetxlog/pg_resetxlog.c      |    2 
 src/include/access/xlog.h                |   14 
 src/include/access/xlog_internal.h       |    4 
 src/include/catalog/pg_control.h         |    3 
 src/include/postmaster/bgwriter.h        |    6 
 src/include/storage/pmsignal.h           |    1 
 src/test/regress/expected/opr_sanity.out |    7 
 18 files changed, 579 insertions(+), 79 deletions(-), 719 modifications(!)

Please review everybody. Many thanks.

-- 
 Simon Riggs           www.2ndQuadrant.com
 PostgreSQL Training, Services and Support

Attachment: recovery_infrastruc.v8.patch
Description: text/x-patch (85.6 KB)

Responses

pgsql-hackers by date

Next:From: Gurjeet SinghDate: 2008-10-01 00:44:43
Subject: Re: FSM rewrite committed, loose ends
Previous:From: Tom LaneDate: 2008-09-30 22:52:15
Subject: WAL recovery is broken by FSM patch

pgsql-patches by date

Next:From: Markus SchaafDate: 2008-10-01 14:13:24
Subject: libpq not linked against libgssapi
Previous:From: Simon RiggsDate: 2008-09-30 07:03:23
Subject: Re: still alive?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group