Skip site navigation (1) Skip section navigation (2)

Re: WAL "low watermark" during base backup

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: Simon Riggs <simon(at)2ndquadrant(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: WAL "low watermark" during base backup
Date: 2011-09-05 10:38:15
Message-ID: CABUevEw1BQ7MsXc_9ur7Oz7sWchV451L89JetYrdMuaqhP1HEQ@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Sun, Sep 4, 2011 at 19:02, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
> On Fri, Sep 2, 2011 at 6:52 PM, Magnus Hagander <magnus(at)hagander(dot)net> wrote:
>
>> Attached patch implements a "low watermark wal location" in the
>> walsender shmem array. Setting this value in a walsender prevents
>> transaction log removal prior to this point - similar to how
>> wal_keep_segments work, except with an absolute number rather than
>> relative. For now, this is set when running a base backup with WAL
>> included - to prevent the required WAL to be recycled away while the
>> backup is running, without having to guestimate the value for
>> wal_keep_segments. (There could be other ways added to set it in the
>> future, but that's the only one I've done for now)
>>
>> It obviously needs some documentation updates as well, but I wanted to
>> get some comments on the way it's done before I work on those.
>
> I'm not yet fully available for a discussion on this, but not sure I like this.
>
> You don't have to guess the setting of wal_keep_segments, you
> calculate it exactly from the size of your WAL disk. No other
> calculation is easy or accurate.

Uh, no. What about the (very large number of) cases where pg is just
sitting on one partition, possibly shared with a whole lot of other
services? You'd need to set it to all-of-your-disk, which is something
that will change over time.

Maybe I wasn't entirely clear in the submission, but if it wasn't
obvious: the use-case for this is the small and simple installations
that need a simple way of doing a reliable online backup. This is the
"pg_basebackup -x" usecase altogether - for example, anybody "bigger"
likely has archiv elogging setup already, in which case this
functionality is not interesting at all.

> This patch implements "fill disk until primary croaks" behaviour which
> means you are making a wild and risky guess as to whether it will
> work. If it does not, you are hosed.

Replace "primary" with "server" - remember that this is about backups
and not replication primarily.

That said, you are correct, it does implement that. But then again,
logging into the database and opening a transaction and just leaving
it around for $forever will have similar problems - yet, we allow
users to do that.

-- 
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/

In response to

Responses

pgsql-hackers by date

Next:From: Heikki LinnakangasDate: 2011-09-05 11:10:54
Subject: Re: WIP: Fast GiST index build
Previous:From: Marti RaudseppDate: 2011-09-05 07:52:44
Subject: Redundant bitmap index scans on smallint column

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group