pgsql: Make pg_receivexlog and pg_basebackup -X stream work across time

From: Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi>
To: pgsql-committers(at)postgresql(dot)org
Subject: pgsql: Make pg_receivexlog and pg_basebackup -X stream work across time
Date: 2013-01-17 18:25:23
Message-ID: E1Tvu9T-00076h-HP@gemulon.postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Make pg_receivexlog and pg_basebackup -X stream work across timeline switches.

This mirrors the changes done earlier to the server in standby mode. When
receivelog reaches the end of a timeline, as reported by the server, it
fetches the timeline history file of the next timeline, and restarts
streaming from the new timeline by issuing a new START_STREAMING command.

When pg_receivexlog crosses a timeline, it leaves the .partial suffix on the
last segment on the old timeline. This helps you to tell apart a partial
segment left in the directory because of a timeline switch, and a completed
segment. If you just follow a single server, it won't make a difference, but
it can be significant in more complicated scenarios where new WAL is still
generated on the old timeline.

This includes two small changes to the streaming replication protocol:
First, when you reach the end of timeline while streaming, the server now
sends the TLI of the next timeline in the server's history to the client.
pg_receivexlog uses that as the next timeline, so that it doesn't need to
parse the timeline history file like a standby server does. Second, when
BASE_BACKUP command sends the begin and end WAL positions, it now also sends
the timeline IDs corresponding the positions.

Branch
------
master

Details
-------
http://git.postgresql.org/pg/commitdiff/0b6329130e8e4576e97ff763f0e773347e1a88af

Modified Files
--------------
doc/src/sgml/protocol.sgml | 14 +-
src/backend/access/transam/timeline.c | 14 +-
src/backend/access/transam/xlog.c | 24 +-
src/backend/access/transam/xlogfuncs.c | 4 +-
src/backend/replication/basebackup.c | 41 ++-
src/backend/replication/walsender.c | 50 ++-
src/bin/pg_basebackup/pg_basebackup.c | 23 +-
src/bin/pg_basebackup/pg_receivexlog.c | 99 +++---
src/bin/pg_basebackup/receivelog.c | 687 ++++++++++++++++++++++---------
src/bin/pg_basebackup/receivelog.h | 2 +-
src/include/access/timeline.h | 3 +-
src/include/access/xlog.h | 6 +-
12 files changed, 677 insertions(+), 290 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Alvaro Herrera 2013-01-17 19:15:45 pgsql: Accelerate end-of-transaction dropping of relations
Previous Message Tom Lane 2013-01-17 18:13:12 pgsql: Improve memory space management in tuplesort and tuplestore.