Re: Remove Deprecated Exclusive Backup Mode

On Thu, Jul 2, 2020 at 1:06 AM Jehan-Guillaume de Rorthais <jgdr(at)dalibo(dot)com>
wrote:

> On Wed, 1 Jul 2020 15:58:57 -0400
> Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> > On Wed, Jul 1, 2020 at 3:50 PM Magnus Hagander <magnus(at)hagander(dot)net>
> wrote:
> > > As far as I've seen, the one thing that people have problems with in
> the
> > > exclusive mode backups are precisely the fact that they have to keep a
> > > persistent conneciton open, and thus it cannot work together with
> backup
> > > software that is limited to only supporting running a pre- and a post
> > > script.
> > >
> > > Something like I have suggested here is to solve *that* problem. I
> don't
> > > think anybody actually explicitly wants "exclusive backups" -- they
> want a
> > > backup solution that plugs into their world of pre/post scripts. And
> if we
> > > can make that one work in a safer way than the current exclusive
> backups,
> > > ohw is that not an improvement?
> >
> > Yeah, I guess that's a pretty fair point. I have to confess to having
> > somewhat limited enthusiasm for adding a third mode here, but it might
> > be worth it.
> >
> > It seems pretty well inevitable to me that people are going to forget
> > to end them.
>
> There's so many way an admin could break their backup process and not
> notice
> it. Even with current nonexclusive backup, a bad written script might
> creates
> bad unrecoverable backups.
>

Definitely. And the scariest part of them is that they all work fine in
testing...

In regard with your concern, monitoring in progress backups might be *one*
> answer, from server side. This was "easy" with exclusive backup, we just
> monitor the backup_label age and warn if it is older than expected [1].
> Using
>

Yeah, but having a monitoring point that will crash your database on
restart isn't very safe :)

> non-exclusive backup...this is not that easy anymore. And
> pg_is_in_backup() is
> quite misleading if the admin found it without reading doc. Maybe an admin
>

Yeah, as it is now it should really be called pg_is_in_exclusive_backup().

> function to list in progress non-exclusive backup and related backend pid
> might
> be a good start?
>

I think it would make perfect sense to show manual (exclusive or
non-exclusive) base backups in pg_stat_progress_basebackup. There's no
fundamental need that one should only be for base backups taken with
pg_basebackup. In fact, I'd argue that's a mistake in the view in v13 that
it does not include this information.

It could have "phase" set to "manual non-exclusive" for example, and leave
the other fields at NULL.

I guess in theory even for exclusive ones, with the pid column set to NULL.
(As Stephen mentioned at some point in the future we might also want to
extend it to allow these tools to report their progress as well into it,
probably by just calling an admin function on the connection that they
already have).

Tools like pg_basebackup (and probably pgbackrest/barman to some extends)
> still
> need the backup to abort on disconnection. Maybe it could flag its session
> using the replication protocol or call a new admin function or load a hook
> on
> session-shutdown to keep previous API behavior?
>

Suddenly requiring a replication protocol connection for one thing, when
all their other things are done over a normal one, seems really terrible.
But having an admin function would work.

But anything requiring loading of a hook is going to raise the bar
massively as suddenly somebody needs to install a compiled binary on the
server in order to use it. But calling a separate function is pretty much
what I suggested upthread (except I suggested a new version of
pg_stop_backup, but implementation wise)

But I'm not sure what previous API behavior you're looking to preserve here?

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>