Skip site navigation (1) Skip section navigation (2)

Re: Teaching pg_receivexlog to follow timeline switches

From: Fujii Masao <masao(dot)fujii(at)gmail(dot)com>
To: Heikki Linnakangas <hlinnakangas(at)vmware(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Teaching pg_receivexlog to follow timeline switches
Date: 2013-01-15 18:22:03
Message-ID: CAHGQGwEzkyyemeEn7T-c-Xm+TcQbAX_GEDMOPAFi5o-fU2Vw-g@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Tue, Jan 15, 2013 at 11:05 PM, Heikki Linnakangas
<hlinnakangas(at)vmware(dot)com> wrote:
> Now that a standby server can follow timeline switches through streaming
> replication, we should do teach pg_receivexlog to do the same. Patch
> attached.
>
> I made one change to the way START_STREAMING command works, to better
> support this. When a standby server reaches the timeline it's streaming from
> the master, it stops streaming, fetches any missing timeline history files,
> and parses the history file of the latest timeline to figure out where to
> continue. However, I don't want to parse timeline history files in
> pg_receivexlog. Better to keep it simple. So instead, I modified the
> server-side code for START_STREAMING to return the next timeline's ID at the
> end, and used that in pg_receivexlog. I also modifed BASE_BACKUP to return
> not only the start XLogRecPtr, but also the corresponding timeline ID.
> Otherwise we might try to start streaming from wrong timeline if you issue a
> BASE_BACKUP at the same moment the server switches to a new timeline.
>
> When pg_receivexlog switches timeline, what to do with the partial file on
> the old timeline? When the timeline changes in the middle of a WAL segment,
> the segment old the old timeline is only half-filled. For example, when
> timeline changes from 1 to 2, you'll have this in pg_xlog:
>
> 000000010000000000000006
> 000000010000000000000007
> 000000010000000000000008
> 000000020000000000000008
> 00000002.history
>
> The segment 000000010000000000000008 is only half-filled, as the timeline
> changed in the middle of that segment. The beginning portion of that file is
> duplicated in 000000020000000000000008, with the timeline-changing
> checkpoint record right after the duplicated portion.
>
> When we stream that with pg_receivexlog, and hit the timeline switch, we'll
> have this situation in the client:
>
> 000000010000000000000006
> 000000010000000000000007
> 000000010000000000000008.partial
>
> What to do with the partial file? One option is to rename it to
> 000000010000000000000008. However, if you then kill pg_receivexlog before it
> has finished streaming a full segment from the new timeline, on restart it
> will try to begin streaming WAL segment 000000010000000000000009, because it
> sees that segment 000000010000000000000008 is already completed. That'd be
> wrong.

Can't we rename .partial file safely after we receive a full segment
of the WAL file
with new timeline and the same logid/segmentid?

Regards,

-- 
Fujii Masao


In response to

Responses

pgsql-hackers by date

Next:From: Andres FreundDate: 2013-01-15 18:30:35
Subject: Re: logical changeset generation v4
Previous:From: Alvaro HerreraDate: 2013-01-15 18:19:43
Subject: Re: recent ALTER whatever .. SET SCHEMA refactoring

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group