Sending logical replication data only after synchronous replication happened

From: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Cc: Ekaterina Vagina <vag-ekaterina(at)yandex-team(dot)ru>, tserakhau – Андрей Терехов <robot-femida(at)yandex-team(dot)ru>
Subject: Sending logical replication data only after synchronous replication happened
Date: 2020-10-16 07:21:27
Message-ID: 03D4D460-A5F9-4AC8-B4BA-B0290671BCD6@yandex-team.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi!

On PgCon 2020 we had been discussing some caveats of synchronous replication [0] related to data durability in HA postgres installations.

But also there was raised important concern about streaming logical replication only after it "actually happened" for HA cluster.
Is anyone working on it?If no, I propose to discuss design of this feature.

Why is it important? It's important for changed data capture (CDC).
For physical replication we can apply changed forward (just replay WAL) and backward (with help of pg_rewind).
But there is no clean way to undo logical replication.

Consider someone having a data publication from HA cluster A to another postgres installation B. A consists of primary A1 and standby A2.

When failover happens from A1 to A2 some part of A1 history can be committed locally on A. And streamed to B via logical replication. After failover to A2 B cannot continue CDC from A2 because B already applied part of a history from A1 which never existed for A2.

During unconference session [0] there was proposed GUC that is 'post_synchronous_standby_names' of standbys that can't get data until the transaction has been sent to the sync standbys.
This will do the trick, though I'm not sure It's best possible interface for the feature.
Any ideas on the feature will be appreciated.

Thanks!

Best regards, Andrey Borodin.

[0] https://wiki.postgresql.org/wiki/PgCon_2020_Developer_Unconference/Edge_cases_of_synchronous_replication_in_HA_solutions

Browse pgsql-hackers by date

  From Date Subject
Next Message Ashutosh Bapat 2020-10-16 07:25:26 Enumize logical replication message actions
Previous Message Bharath Rupireddy 2020-10-16 06:23:25 Re: Parallel Inserts in CREATE TABLE AS