Re: Synchronizing slots from primary to standby

From: Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Synchronizing slots from primary to standby
Date: 2022-01-03 13:46:52
Message-ID: 646acd8d-9896-ed8d-2ab4-0bf11b3285d4@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Here is an updated patch to fix some build failures. No feature changes.

On 14.12.21 23:12, Peter Eisentraut wrote:
> On 31.10.21 11:08, Peter Eisentraut wrote:
>> I want to reactivate $subject.  I took Petr Jelinek's patch from [0],
>> rebased it, added a bit of testing.  It basically works, but as
>> mentioned in [0], there are various issues to work out.
>>
>> The idea is that the standby runs a background worker to periodically
>> fetch replication slot information from the primary.  On failover, a
>> logical subscriber would then ideally find up-to-date replication
>> slots on the new publisher and can just continue normally.
>
>> So, again, this isn't anywhere near ready, but there is already a lot
>> here to gather feedback about how it works, how it should work, how to
>> configure it, and how it fits into an overall replication and HA
>> architecture.
>
> Here is an updated patch.  The main changes are that I added two
> configuration parameters.  The first, synchronize_slot_names, is set on
> the physical standby to specify which slots to sync from the primary. By
> default, it is empty.  (This also fixes the recovery test failures that
> I had to disable in the previous patch version.)  The second,
> standby_slot_names, is set on the primary.  It holds back logical
> replication until the listed physical standbys have caught up.  That
> way, when failover is necessary, the promoted standby is not behind the
> logical replication consumers.
>
> In principle, this works now, I think.  I haven't made much progress in
> creating more test cases for this; that's something that needs more
> attention.
>
> It's worth pondering what the configuration language for
> standby_slot_names should be.  Right now, it's just a list of slots that
> all need to be caught up.  More complicated setups are conceivable.
> Maybe you have standbys S1 and S2 that are potential failover targets
> for logical replication consumers L1 and L2, and also standbys S3 and S4
> that are potential failover targets for logical replication consumers L3
> and L4.  Viewed like that, this setting could be a replication slot
> setting.  The setting might also have some relationship with
> synchronous_standby_names.  Like, if you have synchronous_standby_names
> set, then that's a pretty good indication that you also want some or all
> of those standbys in standby_slot_names.  (But note that one is slots
> and one is application names.)  So there are a variety of possibilities.

Attachment Content-Type Size
v3-0001-Synchronize-logical-replication-slots-from-primar.patch text/plain 55.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message tushar 2022-01-03 14:09:58 Re: refactoring basebackup.c
Previous Message Peter Eisentraut 2022-01-03 13:18:35 Re: Add Boolean node