Re: Single transaction in the tablesync worker?

From: Peter Smith <smithpb2250(at)gmail(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Single transaction in the tablesync worker?
Date: 2021-01-12 11:53:51
Message-ID: CAHut+PsPO2vOp+P7U2szROMy15PKKGanhUrCYQ0ffpy9zG1V1A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Amit.

PSA the v14 patch for the Tablesync Solution1.

Main differences from v13:
+ Addresses all review comments 1-5, posted 9/Jan [ak9]
+ Addresses review comment 1, posted 11/Jan [ak11]
+ Modifications per suggestion [ak11] to handle race scenarios during
Drop/AlterSubscription
+ Changed LOG to WARNING if DropSubscription unable to drop tablesync slot

[ak9] = https://www.postgresql.org/message-id/CAA4eK1%2BgUBxKcYWg%2BMCC6Qbw-My%2B2wKUct%2BiFtr-_HgundUUBQ%40mail.gmail.com
[ak11] = https://www.postgresql.org/message-id/CAA4eK1KGUt86A7CfuQW6OeDvAhEbVk8VOBJmcoZjrYBn965kOA%40mail.gmail.com

====

Features:

* The tablesync slot is now permanent instead of temporary.

* The tablesync slot name is no longer tied to the Subscription slot name.

* The tablesync slot cleanup (drop) code is added for
DropSubscription, AlterSubscription_refresh and for
process_syncing_tables_for_sync functions. Drop/AlterSubscription will
issue WARNING instead of ERROR in case the slot drop fails.

* The tablesync worker is now allowing multiple tx instead of single tx

* A new state (SUBREL_STATE_FINISHEDCOPY) is persisted after a
successful copy_table in tablesync's LogicalRepSyncTableStart.

* If a re-launched tablesync finds state SUBREL_STATE_FINISHEDCOPY
then it will bypass the initial copy_table phase.

* Now tablesync sets up replication origin tracking in
LogicalRepSyncTableStart (similar as done for the apply worker). The
origin is advanced when first created.

* The tablesync replication origin tracking is cleaned up during
DropSubscription and/or process_syncing_tables_for_apply.

* The DropSubscription cleanup code was enhanced (v7+) to take care of
any crashed tablesync workers.

* The AlterSubscription_refresh (v14+) is now more similar to
DropSubscription w.r.t to stopping workers for any "removed" tables.

* Updates to PG docs.

TODO / Known Issues:

* Minor review comments

===

Also PSA some detailed logging evidence of some test scenarios
involving Drop/AlterSubscription:
+ Test-20210112-AlterSubscriptionRefresh-ok.txt =
AlterSubscription_refresh which successfully drops a tablesync slot
+ Test-20210112-AlterSubscriptionRefresh-warning.txt =
AlterSubscription_refresh gives WARNING that it cannot drop the
tablesync slot (which no longer exists)
+ Test-20210112-DropSubscription-warning.txt = DropSubscription with a
disassociated slot_name gives a WARNING that it cannot drop the
tablesync slot (due to broken connection)

---
Kind Regards,
Peter Smith.
Fujitsu Australia

Attachment Content-Type Size
Test-20210112-DropSubscription-warning.txt text/plain 21.8 KB
v14-0002-Tablesync-extra-logging.patch application/octet-stream 7.8 KB
v14-0001-Tablesync-Solution1.patch application/octet-stream 39.0 KB
Test-20210112-AlterSubscriptionRefresh-warning.txt text/plain 38.3 KB
Test-20210112-AlterSubscriptionRefresh-ok.txt text/plain 20.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Smith 2021-01-12 12:13:04 Re: Single transaction in the tablesync worker?
Previous Message japin 2021-01-12 11:53:24 Re: Logical Replication - behavior of ALTER PUBLICATION .. DROP TABLE and ALTER SUBSCRIPTION .. REFRESH PUBLICATION