Re: Teaching pg_receivexlog to follow timeline switches

On Tue, Jan 15, 2013 at 11:05 PM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> Now that a standby server can follow timeline switches through streaming
> replication, we should do teach pg_receivexlog to do the same. Patch
> attached.
>
> I made one change to the way START_STREAMING command works, to better
> support this. When a standby server reaches the timeline it's streaming from
> the master, it stops streaming, fetches any missing timeline history files,
> and parses the history file of the latest timeline to figure out where to
> continue. However, I don't want to parse timeline history files in
> pg_receivexlog. Better to keep it simple. So instead, I modified the
> server-side code for START_STREAMING to return the next timeline's ID at the
> end, and used that in pg_receivexlog. I also modifed BASE_BACKUP to return
> not only the start XLogRecPtr, but also the corresponding timeline ID.
> Otherwise we might try to start streaming from wrong timeline if you issue a
> BASE_BACKUP at the same moment the server switches to a new timeline.
>
> When pg_receivexlog switches timeline, what to do with the partial file on
> the old timeline? When the timeline changes in the middle of a WAL segment,
> the segment old the old timeline is only half-filled. For example, when
> timeline changes from 1 to 2, you'll have this in pg_xlog:
>
> 000000010000000000000006
> 000000010000000000000007
> 000000010000000000000008
> 000000020000000000000008
> 00000002.history
>
> The segment 000000010000000000000008 is only half-filled, as the timeline
> changed in the middle of that segment. The beginning portion of that file is
> duplicated in 000000020000000000000008, with the timeline-changing
> checkpoint record right after the duplicated portion.
>
> When we stream that with pg_receivexlog, and hit the timeline switch, we'll
> have this situation in the client:
>
> 000000010000000000000006
> 000000010000000000000007
> 000000010000000000000008.partial
>
> What to do with the partial file? One option is to rename it to
> 000000010000000000000008. However, if you then kill pg_receivexlog before it
> has finished streaming a full segment from the new timeline, on restart it
> will try to begin streaming WAL segment 000000010000000000000009, because it
> sees that segment 000000010000000000000008 is already completed. That'd be
> wrong.

Can't we rename .partial file safely after we receive a full segment
of the WAL file
with new timeline and the same logid/segmentid?

Regards,

--
Fujii Masao