Re: [patch] reorder tablespaces in basebackup tar stream for backup_label

From: Michael Banck <michael(dot)banck(at)credativ(dot)de>
To: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
Cc: Bernd Helmle <mailings(at)oopsware(dot)de>, PostgreSQL Development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [patch] reorder tablespaces in basebackup tar stream for backup_label
Date: 2017-03-16 18:38:10
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


sorry, it took me a while to get back to this.

Am Freitag, den 03.03.2017, 15:44 +0900 schrieb Michael Paquier:
> On Wed, Feb 22, 2017 at 9:23 PM, Bernd Helmle <mailings(at)oopsware(dot)de> wrote:
> > The comment in the code says explicitely to add the base directory to
> > the end of the list, not sure if that is out of a certain reason.
> >
> > I'd say this is an oversight in the implementation. I'm currently
> > working on a tool using the streaming protocol directly and i've
> > understood it exactly the way, that the default tablespace is the first
> > one in the stream.
> >
> > So +1 for the patch.
> Commit 507069de has switched the main directory from the beginning to
> the end of the list, and the thread about this commit is here:
> + /* Add a node for the base directory at the beginning. This way, the
> + * backup_label file is always the first file to be sent. */
> ti = palloc0(sizeof(tablespaceinfo));
> ti->size = opt->progress ? sendDir(".", 1, true, tablespaces,
> true) : -1;
> - tablespaces = lappend(tablespaces, ti);
> + tablespaces = lcons(ti, tablespaces);
> So, the main directory is located at the end on purpose. When using
> --wal-method=fetch the WAL segments are part of the main tarball, so
> if you send the main tarball first you would need to generate a second
> tarball with the WAL segments that have been generated between the
> moment the main tarball has finished until the end of the last
> tablespace taken if you want to have a consistent backup.

Ah, thanks for pointing that out, I've missed that in my testing.

> Your patch would work with the stream mode though.

Or, if not requesting the "WAL" option of the replication protocol's
BASE_BACKUP command.

I agree it doesn't make sense to start messing with fetch mode, but I
don't think we guarantee any ordering of tablespaces (to wit, Bernd was
pretty sure it was the other way around all the time), neither do I
think having the main tablespace be the first for non-WAL/stream and the
last for WAL/fetch would be confusing personally, though I understand
this is debatable.

So I've updated the patch to only switch the main tablespace to be first
in case WAL isn't included, please find it attached.


Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Attachment Content-Type Size
0001-Reorder-tablespaces-for-streaming-basebackups-v2.patch text/x-patch 1.8 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Borodin 2017-03-16 18:53:18 Re: Review: GIN non-intrusive vacuum of posting tree
Previous Message Dilip Kumar 2017-03-16 17:50:42 Re: Parallel Bitmap scans a bit broken