Re: base backup client as auxiliary backend process

From: Alexandra Wang <lewang(at)pivotal(dot)io>
To: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>
Cc: Sergei Kornilov <sk(at)zsrv(dot)org>, Michael Paquier <michael(at)paquier(dot)xyz>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: base backup client as auxiliary backend process
Date: 2020-01-09 10:57:52
Message-ID: CACiyaSqci=MhdEwVZEf=tUOdHfoEZPK9330ajhg5b750nPEenw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Peter,

On Fri, Nov 22, 2019 at 6:22 PM Peter Eisentraut <
peter(dot)eisentraut(at)2ndquadrant(dot)com> wrote:

> Here is another patch set that implements the temporary slot use on the
> walreceiver side, essentially mirroring what pg_basebackup already does.
>
> I think this patch set might be useful on its own, even without the base
> backup stuff to follow.

I very much like this idea of every replication connection should have a
replication slot, either permanent or temporary if user didn't specify. I
agree
that this patch is useful on its own.

> This makes a whole bunch of things much nicer: The connection
> information for where to get the base backup from comes from
> postgresql.conf, so you only need to specify it in one place.
> pg_basebackup is completely out of the picture; no need to deal with
> command-line options, --recovery-conf, screen, monitoring for
> completion, etc. If something fails, the base backup process can
> automatically be restarted (maybe). Operating system integration is
> much easier: You only call initdb and then pg_ctl or postgres, as you
> are already doing. Automated deployment systems don't need to wait for
> pg_basebackup to finish: You only call initdb, then start the server,
> and then you're done -- waiting for the base backup to finish can be
> done by the regular monitoring system.

Back to the base backup stuff, I don't quite understand all the benefits you
mentioned above. It seems to me the greatest benefit with this patch is that
postmaster takes care of pg_basebackup itself, which reduces the human wait
in
between running the pg_basebackup and pg_ctl/postgres commands. Is that
right?
I personally don't mind the --write-recovery-conf option because it helps me
write the primary_conninfo and primary_slot_name gucs into
postgresql.auto.conf, which to me as a developer is easier than editing
postgres.conf without automation. Sorry about the dumb question but what's
so
bad about --write-recovery-conf? Are you planning to completely replace
pg_basebackup with this? Is there any use case that a user just need a
basebackup but not immediately start the backend process?

Also the patch doesn't apply to master any more, need a rebase.

--
Alexandra

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dagfinn Ilmari Mannsåker 2020-01-09 11:03:23 Re: Fixing parallel make of libpq
Previous Message Amit Kapila 2020-01-09 10:51:38 Re: logical decoding : exceeded maxAllocatedDescs for .spill files