Skip site navigation (1) Skip section navigation (2)

Re: pg_basebackup and wal streaming

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: pg_basebackup and wal streaming
Date: 2011-02-26 17:19:27
Message-ID: AANLkTimbvgR_CKabp2j=aD+Ma89B3VvX5anO1x6F97qz@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sat, Feb 26, 2011 at 16:28, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
> On Sun, Feb 20, 2011 at 21:37, Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr> wrote:
>> Hi,
>>
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>> Better late than never (or?), here's the final cleanup of
>>> pg_streamrecv for moving into the main distribution, per discussion
>>> back in late dec or early jan. It also includes the "stream logs in
>>> parallel to backup" part that was not completed on pg_basebackup.
>>
>> And that's something I've been so interested in!  It's only fair game
>> that I spend time reviewing after my insisting for having it :)
>>
>> The programs (pg_basebackup and pg_receivexlog) both work as expected,
>> and show in the pg_stat_replication system view.
>>
>> pg_basebackup -x option should be revised so that it's easier to make
>> the difference between streaming WAL while the base backup is ongoing
>> and fetching them at the end, taking the risk to corrupt the whole
>> backup as soon as wal_keep_segments is undersized.
>>
>>  -x, --xlog[=stream]       include required WAL files in backup
>>
>> It could be --xlog=stream|fetch or something that reads better.
>
> Yeha, that's probably true. I wanted to avoid making it mandatory, but
> it's actually easier this way. Will change it to that.
>
>
>> Now, on to the code itself.
>>
>> I wonder if the segment_callback() routine would better be a while{}
>> loop rather than a recursive construct.  Also, it looks like a lib
>> function but it's doing exit(1)…
>
> Actually, it's even better to just reorder the checks in the other
> order - that way we don't need a loop *or* a self-call.
>
>
>> Unfortunately I can't comment (or won't risk learning enough details
>> tonight to try to be smart here) on FindStreamingStart() implementation,
>> that seems crucial.
>
> It is - so if you can find the time to, that would be great...


Attached is an updated version of the patch that includes these
changes, as well as Windows support and an initial cut at a ref page
for pg_receivexlog (needs some more detail still).


-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

Attachment: streaming_xlog.patch
Description: text/x-patch (51.5 KB)

In response to

Responses

pgsql-hackers by date

Next:From: Bruce MomjianDate: 2011-02-26 17:49:28
Subject: Re: TODO: You can alter it, but you can't view it
Previous:From: Tom LaneDate: 2011-02-26 17:18:46
Subject: Re: wCTE: why not finish sub-updates at the end, not the beginning?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group