Re: BUG #7500: hot-standby replica crash after an initial rsync

I believe I just hit this same issue, but with PG 9.1.3:

<@:32407> 2012-08-29 10:02:09 UTC LOG: shutting down
<@:32407> 2012-08-29 10:02:09 UTC LOG: database system is shut down
<[unknown](at)[unknown]:31687> 2012-08-29 13:34:03 UTC LOG: connection
received: host=[local]
<[unknown](at)[unknown]:31687> 2012-08-29 13:34:03 UTC LOG: incomplete
startup packet
<@:31686> 2012-08-29 13:34:03 UTC LOG: database system was
interrupted; last known up at 2012-08-29 13:14:47 UTC
<@:31686> 2012-08-29 13:34:03 UTC LOG: entering standby mode
<@:31686> 2012-08-29 13:34:03 UTC LOG: redo starts at A92/5F000020
<@:31686> 2012-08-29 13:34:03 UTC FATAL: could not access status of
transaction 208177034
<@:31686> 2012-08-29 13:34:03 UTC DETAIL: Could not read from file
"pg_multixact/offsets/0C68" at offset 131072: Success.
<@:31686> 2012-08-29 13:34:03 UTC CONTEXT: xlog redo create multixact
208177034 offset 1028958730: 1593544329 1593544330
<@:31681> 2012-08-29 13:34:03 UTC LOG: startup process (PID 31686)
exited with exit code 1
<@:31681> 2012-08-29 13:34:03 UTC LOG: terminating any other active
server processes

This was attempting to rebuild a hot standby after switching my master
to a new server. In between the shutdown and the attempt to restart:

- The master was put into backup mode.
- The datadir was rsynced over, using rsync -ahhP --delete-before
--exclude=postmaster.pid --exclude=pg_xlog
- The master was taken out of backup mode.
- The pg_xlog directory was emptied
- The pg_xlog directory was rsynced across from the master. This
included all the WAL files from before the promotion, throughout
backup mode, and a few from after backup mode was left.
- recovery.conf was created (hot_standby=on, primary_conninfo=...,
recovery_target_timeline=latest)
- The hot standby was started.

Unfortunately I didn't have the disk space around to keep everything,
and am attempting the rebuild again now.

--
Stuart Bishop <stuart(at)stuartbishop(dot)net>
http://www.stuartbishop.net/