Hot Standby 0.2.1

From: Simon Riggs <simon(at)2ndQuadrant(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Hot Standby 0.2.1
Date: 2009-09-15 21:41:59
Message-ID: 1253050920.27962.60.camel@ebony.2ndQuadrant
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

OK, here is the latest version of the Hot Standby patchset. This is
about version 30+ by now, but we should regard this as 0.2.1
Patch against CVS HEAD (now): clean apply, compile, no known bugs.


You can download PDF versions of the fine manual is here

Also available via the project Wiki, which is here

Patch should be attached to this email. If problems, get this and other
versions from Wiki please. No offlist comments, questions etc please.


0 meaning its not fully released yet
2 meaning this is a major new re-write
1 meaning this is the first release

Patch is still in testing and will be for next few days at least.
Released now only because I promised to do so.

Is this ready for commit? Yes, it is in the shape I want it to be in,
but also, No, I can't say it's been through a wide enough range of tests
as yet to be considered immediately ready for commit.

Further bug fixing and minor cosmetic development will take place via my
GIT repo, uploaded soon. Patch included here to meet deadlines and code
inclusion. BSD.


* 76 files changed, 5160 insertions(+), 59 deletions(-), 1251 mods(!)
* 7 files with more than 100 lines changed
procarray.c (1200+ additions)
xlog.c (600+ additions)
xact.c (500+ additions)
inval.c (650 additions)
* 29 files with 10 or fewer lines changed
* Applies cleanly to CVS HEAD as of now


The rough changes since version 1 series of patches.

* Full documentation included. Many, but not all nuances of SGML tagging
followed, but sufficient aspects there to allow for proofreading before
we do final changes. Some undocumented functions now correctly
documented. Recovery functions now split into user and control functions
in docs to make it clearer.

* Starting conditions in GetRunningTransactionData() are now much
stricter and holds more lwlocks. There are few cases where any not-found
xids are allowed during xid processing, so code is more robust. Please
check for race conditions.

* GetRunningTransactionData() now handles initialisation of
AccessExclusiveLocks correctly. Locks are counted in a low-contention
approach that avoids taking holding lock partition locks, if possible.

* Start-up conditions now recoded to allow faster start in cases where
many subtransactions are present. Recovery connections are only enabled
when the snapshot is valid.

* max_connections needs to be correctly set or HS will not allow
connections. Once snapshots are enabled they will continue to be
available always.

* RecordKnownAssignedTransactions() now contains a test for xid
wraparound threat which invokes conflict processing should that occur.

* Boolean states now clarified and corrected. Hot Standby can be turned
off completely if not required or if problems effect production. That
causes many changes but there is no change in the intention of those
sections of code.

* UnobservedXids processing follows Heikki's proposal, but has been
renamed to KnownAssignedXids. It has also been modularised and
completely re-written using a hash table approach. So far it has been
much more stable than the previous sorted array coding, which I am happy
to see in the shredder for all the problems it caused. Fully detailed
comments all through.

* All record types now respect max_standby_delay.

* All deferred conflict processing has been removed - conflict
processing itself is still enabled.

* A few other functions have been renamed and/or moved around to
rationalise their exact purpose/position within their modules.

* Prepared transactions holding AccessExclusiveLocks at the end of
recovery are now handled.

* Hash indexes are now safely handled. That was removed at request, but
we need it to avoid silent data loss for queries near those types of

* Hint bits are now set, in appropriate circumstances.

* Flat file logic removed

* Large swathes of unused code removed.

* All code comments addressed and/or re-explained in more detail

* CHECKPOINT now works during recovery but performs restartpoint

* Tweaked max_standby_delay code to avoid long duration waits. Added
dynamic function to control delay during recovery. Added code for stats
collection and ps display. Set default to sensible production values.


* Found and fixed missing relcache init file invalidation

* Found and fixed more serious VACUUM FULL-related weirdness <sigh>

* Recently discovered bug has resulted in changes in
XidInMVCCSnapshot(). Heikki's earlier approach did not correctly allow
for the maximum size of a snapshot. The simplicity of Heikki's proposal
is good, but hid a flaw in where snapshots would put their xids. I've
fully solved the problem though I expect further discussion.

I've looked through every change and verified it, but fixing all the
bugs means there's areas of new code added in last few days. I accept
that any bugs herein are my responsibility.

Simon Riggs

Attachment Content-Type Size
hot_standby.riggs.v0.2.1.patch.bz2 application/x-bzip 70.8 KB


Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2009-09-15 21:42:05 Re: WIP: generalized index constraints
Previous Message Robert Haas 2009-09-15 21:31:13 Re: I need a Postgres Admin $130K + 20K in NYC Any Ideas?