Re: Allowing multiple concurrent base backups

On Tue, Jan 11, 2011 at 19:51, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com> writes:
>> I implemented this in two ways, and can't decide which I like better:
>
>> 1. The contents of the backup label file are returned to the caller of
>> do_pg_start_backup() as a palloc'd string.
>
>> 2. do_pg_start_backup() creates a temporary file that the backup label
>> is written to (instead of "backup_label").
>
>> Implementation 1 changes more code, as pg_start/stop_backup() need to be
>> changed to write/read from memory instead of file, but the result isn't
>> any more complicated. Nevertheless, I somehow feel more comfortable with 2.
>
> Seems like either one of these is fairly problematic in that you have to
> have some monstrous kluge to get the backup_label file to appear with
> the right name in the tarfile.  How badly do we actually need this?
> I don't think the use-case for concurrent base backups is all that large
> in practice given the I/O hit it's going to involve.

I think it can be done cleaner in the tar file injection - I've been
chatting with Heikki offlist about that. Not sure, but I have a
feeling it does.

As for the use-case, it doesn't necessarily involve a huge I/O hit if
either the entire DB is in RAM (not all that uncommon - though then
the backup finishes quickly as well), or if the *client* is slow, thus
making the server backlogged on sending the base backup.

Having it possible to do it concurrently also makes for *much* easier
use of the feature. It might be just about overlapping with a few
seconds, for example - which would probably have no major effect, but
takes a lot more code on the other end to work around.

--
 Magnus Hagander
 Me: http://www.hagander.net/
 Work: http://www.redpill-linpro.com/