Re: Updated backup APIs for non-exclusive backups

From: David Steele <david(at)pgmasters(dot)net>
To: Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>, Stephen Frost <sfrost(at)snowman(dot)net>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Bruce Momjian <bruce(at)momjian(dot)us>, Fujii Masao <masao(dot)fujii(at)gmail(dot)com>, Magnus Hagander <magnus(at)hagander(dot)net>, Marco Nenciarini <marco(dot)nenciarini(at)2ndquadrant(dot)it>, Noah Misch <noah(at)leadboat(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>
Subject: Re: Updated backup APIs for non-exclusive backups
Date: 2018-11-26 14:45:49
Message-ID: a31088f4-2251-8f53-8565-f39f651a3d68@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 11/26/18 12:31 AM, Laurenz Albe wrote:
> On Sun, 2018-11-25 at 16:04 -0500, Stephen Frost wrote:
>>>> There isn’t any need to write the backup label before you restore the database,
>>>> just as you write recovery.conf then.
>>>
>>> Granted.
>>> But it is pretty convenient, and writing it to the data directory right away
>>> is a good thing on top, because it reduces the danger of inadvertedly
>>> starting the backup without recovery.
>>
>> Writing it into the data directory is *not* a good thing because as soon as you do
>> that you risk there being an issue if there’s a crash. Writing into the backup
>> isn’t a bad idea but if you’re managing your backups then writing it somewhere else
>> (such as where you write your WAL) and associating it with the backup (presumably
>> it has a label) should make it easy to pull back when you restore.
>
> If there is a crash during the backup procedure, the backup is bad.
> Doesn't matter during which part of the backup procedure it happens.

Yes, but in this case with exclusive backups your cluster also will not
start. That's a lot different than a bad backup because you should have
many backups and a way to mark when they are complete. If your cluster
requires manual intervention to start because a backup was in progress
that's not a good thing.

I'm of the opinion that we should remove the exclusive option for PG12.
The recovery changes recently committed will completely break any
automated recovery written for PG <= 11 so we might as go all the way.

Requiring non-exclusive backup will be an easy change for tools that
already understand non-exclusive backup, which has been available in the
last three versions of Postgres.

The exclusive backup method tries to make a hard problem look simple but
does not succeed and causes issues besides. Performing backups properly
*is* hard and we don't do anyone a service by making it look like
something can be done with a trivial bash script.

Regards,

--
-David
david(at)pgmasters(dot)net

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2018-11-26 15:02:27 Re: Updated backup APIs for non-exclusive backups
Previous Message Tom Lane 2018-11-26 14:39:50 Re: Doc patch on psql output formats