Re: Deriving Recovery Snapshots

On Tue, 2008-10-14 at 18:50 +0100, Simon Riggs wrote:

> I've worked out what I think is a workable, efficient process for
> deriving snapshots during recovery. I will be posting a patch to show
> how this works tomorrow [Wed 15 Oct], just doing cleanup now.

OK, here's the latest patch. Found a bug late last night, fixed now.

This patch derives snapshot data during recovery. It contains all code
to write and read back WAL changes etc.. It is the "main patch" for Hot
Standby. Everything else hangs around this, enhances it or puts in
additional checks.

Having said that, this patch doesn't let you connect and run queries.
I've written this part as a standalone patch, for easier reviewing. So
it is still a "WIP" patch.

Patch can be tested by running a workload and then stop server, -m
immediate and then watching the replay log (or gdb). Successful tested
with some custom subtransaction scripts and lots of tests using make
installcheck-parallel, then crash recovery.

diffstat
backend/access/transam/slru.c | 16
backend/access/transam/twophase.c | 2
backend/access/transam/xact.c | 664 +++++++++++++++++++!!!!!!!!
backend/access/transam/xlog.c | 58 +-
backend/storage/ipc/procarray.c | 781 +++++++++++++++++++++++++++!!!
backend/storage/lmgr/proc.c | 107 +++++
backend/utils/time/tqual.c | 27 !
include/access/xact.h | 26 +
include/access/xlog.h | 44 +!
include/access/xlog_internal.h | 2
include/catalog/pg_control.h | 3
include/storage/proc.h | 4
include/storage/procarray.h | 17
include/utils/snapshot.h | 65 +++
14 files changed, 1541 insertions(+), 19 deletions(-), 256 mods(!)

Prepared transactions do not yet work correctly, but the principles are
the same so I expect this to be done in a similar way. Any input
welcome.

Other related patches are
* recovery_infrastruc.v9.patch
* atomic_subxids.v7.patch
They don't all apply cleanly together, but the changes are unrelated, so
those patches can still be reviewed without wasting energy.

Next phase is connecting and running queries, next few days. That will
probably shake out a few more bugs from this code.

Comments welcome.

--
Simon Riggs www.2ndQuadrant.com
PostgreSQL Training, Services and Support