Re: Updated backup APIs for non-exclusive backups

On Sun, Nov 25, 2018 at 9:45 PM Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>
wrote:

> Stephen Frost wrote:
> > > > Lastly, if you really want, you can extract out the data from
> > > > pg_stop_backup in whatever your post-backup script is.
> > >
> > > Come on, now.
> > > You usually use backup techniques like that because you can't get
> > > your large database backed up in the available time window otherwise.
> >
> > I’m not following what you’re trying to get at here, why can’t you
> extract
> > the data for the backup label from pg_stop_backup..? Certainly other
> tools
> > do, even ones that do extremely fast parallel backups.. the two are
> > completely independent.
> >
> > Did you think I meant pg_basebackup..? I certaily didn’t.
>
> Oh yes, I misunderstood. Sorry.
>
> Yes, you can come up with a post-backup script that somehow communicates
> with your pre-backup script to get the information, but it sure is
> inconvenient. Simplicity is good in backup solutions, because complicated
> things tend to break more easily.
>

Yes, simplicity is good.

The problem with the previous interface is that it made things *look*
simple, but they were not. Thus, people got into all sorts of trouble
because they got things wrong.

The new interface is simple, and much harder to get wrong. What it isn't,
is that it's not as convenient as the old one. But correctness is a lot
more important than convenience.

That said, I agree with your point that it's not an uncommon thing to need.
If a good solution for it can be proposed that requires the exclusive
backup interface, then I wouldn't be against un-deprecating that. But
that's going to require a lot more than just a documentation change, IMHO.
What could perhaps be handled with a documentation change, however, is to
document a good way for this type of setup to use the new interfaces.

> > I thought our goal is to provide convenient backup methods...
> >
> > Correctness would be first and having a broken system because of a crash
> during a backup isn’t correct.
>
> "Not starting up without manual intervention" is not actually broken...
>

Correct. But that's not the worst case scenario here. Yes, some of the bad
ones are the ones amplified by said manual intervention, but user
interaction at the restore point is going to be even harder to get right.

I'm arguing on behalf of users that run a few databases, want a simple
> backup
> solution and are ready to deal with the shortcomings.
>

Those that want a simple backup solution have one -- pg_basebackup.

The exclusive backup API is *not* simple. It is convenient, but it is not
simple.

Actually having a simple API that worked with the pre/post backup scripts,
that would be an improvement that we should definitely want!

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>