Skip site navigation (1) Skip section navigation (2)

Re: WAL "low watermark" during base backup

From: Florian Pflug <fgp(at)phlo(dot)org>
To: Magnus Hagander <magnus(at)hagander(dot)net>
Cc: Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr>, Simon Riggs <simon(at)2ndquadrant(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WAL "low watermark" during base backup
Date: 2011-09-09 12:40:21
Message-ID: 5286314D-5005-4A19-BBF0-C0D8D906BDCC@phlo.org (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sep9, 2011, at 13:48 , Magnus Hagander wrote:
> On Fri, Sep 9, 2011 at 13:40, Dimitri Fontaine <dimitri(at)2ndquadrant(dot)fr> wrote:
>> Magnus Hagander <magnus(at)hagander(dot)net> writes:
>>>> If you must have this then make pg_basebackup copy xlog files
>>>> regularly during the backup. That way your backup can take forever and
>>>> your primary disk won't fill up. In many cases it actually will take
>>>> forever, but at least we don't take down the primary.
>>> 
>>> There is a patch to do something like that as well sitting on the CF
>>> page. I don't believe one necessarily excludes the other.
>> 
>> I'm not getting why we need the later one when we have this older one?
> 
> One of them is for the simple case. It requires a single connection to
> the server, and it supports things like writing to tarfiles and
> compression.
> 
> The other one is more compelx. It uses multiple connections (one for
> the base, one for the xlog), and as such doesn't support writing to
> files, only directories.

I guess the real question is, why can't we stream the WALs as they are
generated instead of at the end even over a single connection and when
writing tarfiles?

Couldn't we send all available WAL after each single data-file instead
of waiting for all data files to be transferred before sending WAL?

best regards,
Florian Pflug


In response to

Responses

pgsql-hackers by date

Next:From: Dimitri FontaineDate: 2011-09-09 13:53:45
Subject: Re: WAL "low watermark" during base backup
Previous:From: Peter EisentrautDate: 2011-09-09 12:02:14
Subject: Re: regress test failed

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group