Streaming replication failover with 3 servers

From: Stuart Bishop <stuart(at)stuartbishop(dot)net>
To: pgsql-admin(at)postgresql(dot)org
Subject: Streaming replication failover with 3 servers
Date: 2012-08-08 15:51:15
Message-ID: CADmi=6N3qc6q=b=N=wxdGixkx9o-7+ccKS=DD1OXTThVzfk1YQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-admin

Hi.

I'm trying to work out failover and disaster recovery procedures for a
cluster of three servers. Streaming replication is being used with a high
wal_keep_segments, no log shipping is happening. I need to avoid the
several hours it takes to rebuild a hot standby from scratch.

ServerA is the master.
ServerB is a streaming hot standby and prefered failover server.
ServerC is a streaming hot standby.

For a planned failover, maintenance on ServerA:

1. Shutdown ServerB & ServerC
2. Shutdown ServerA
3. Copy pg_xlog from ServerA to ServerB and ServerC
4. Reconfigure ServerB as master, start it up.
5. Reconfigure ServerC as streaming hot standby of ServerB. Start it.
6. After maintenance, reconfigure ServerA as streaming hot standby
of ServerB. Start it.

For an unplanned failover, ServerA has exploded:

1. Run 'SELECT pg_last_xlog_receive_location()' on ServerB and ServerC,
determining which is most up to date.
2. Shutdown ServerB and ServerC
3. If ServerC is more up to date, copy pg_xlog from ServerC to ServerB.
4. Reconfigure ServerB as master, start it up.
5. Reconfigure ServerC as streaming hot standby of ServerB, start it up.

Does this look correct to people?

Am I going to end up in trouble copying files into pg_xlog like this on a
busy system?

Is it overengineered? eg. will a master ensure everything is streamed to
connected hot standbys before a graceful shutdown?

--
Stuart Bishop <stuart(at)stuartbishop(dot)net>
http://www.stuartbishop.net/

Responses

Browse pgsql-admin by date

  From Date Subject
Next Message Amador Alvarez 2012-08-08 17:34:54 Re: standby with a fixed lag behind the master
Previous Message Craig Ringer 2012-08-08 06:57:05 Re: Data migration to sql server 2008