Updated backup APIs for non-exclusive backups

From: Magnus Hagander <magnus(at)hagander(dot)net>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Updated backup APIs for non-exclusive backups
Date: 2016-02-10 12:46:05
Message-ID: CABUevEziPkz6dXOS4knwppPUtDv5Vwnk=OD1nLdFze8NhF8CcA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Per discussionat the developer meeting in Brussels, here's a patch that
makes some updates to the backup APIs, to support non-exclusive backups
without using pg_basebackup. The idea is to fix at least three main issues
that are there today -- that you cannot run concurrent backups, that the
backup_label file is created in the data directory which makes it
impossible to distinguish between a cluster restored from backup and one
that crashed while a backup was running, and a cluster can get "stuck" in
backup mode if the backup script/software crashes.

To make this work, this patch:

* Introduces a new argument for pg_start_backup(), for "exclusive". It
defaults to true, which means that just calling pg_start_backup() like
before will behave exactly like before. If it's called with exclusive='f',
a non-exclusive backup is started, and the backup label is collected in
memory instead of in the backup_label file.

* If the client disconnects with a non-exclusive backup running, the backup
is automatically aborted. This is the same thing that pg_basebackup does.
To use these non-exclusive backups the backup software will have to
maintain a persistent connection to the database -- something that should
not be a problem for any of the modern ones out there (but would've been
slightly trickier back in the days when we suggested shellscripts)

* A new version of pg_stop_backup is created, taking an argument specifying
if it's exclusive or not. The current version of pg_stop_backup() continues
to work for exclusive backups, with no changes to behavior. The new
pg_stop_backup will return a record of three columns instead of just the
value -- the LSN (pglsn), the backup label file (text) and the tablespace
map file (text). If used to cancel an exclusive backup, backup label file
and tablespace map will be NULL. At this point it's up to the backup
software to write down the files in the location of the backup.

Attached patch currently doesn't include full documentation, since Bruce
was going to restructure that section of the docs (also based on the
devmeeting). I will write up the docs once that is done (I assume it will
be soon enough, or I'll go do it regardless), but I wanted to get some
review in on the code while waiting.

--
Magnus Hagander
Me: http://www.hagander.net/
Work: http://www.redpill-linpro.com/

Attachment Content-Type Size
backup_api.patch text/x-patch 18.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2016-02-10 13:10:02 Re: Updated backup APIs for non-exclusive backups
Previous Message Ashutosh Bapat 2016-02-10 12:12:36 Re: postgres_fdw join pushdown (was Re: Custom/Foreign-Join-APIs)