RE: Perform streaming logical transactions by background workers and parallel apply

From: "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-06-08 07:12:30
Message-ID: OS3PR01MB6275208A2F8ED832710F65E09EA49@OS3PR01MB6275.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thur, Jun 2, 2022 6:02 PM I wrote:
> Attach the new patches.

I tried to improve the patches by following 2 points:

1. Improved the patch as suggested by Amit-san that I mentioned in [1].
When publisher sends a "STREAM ABORT" message to subscriber, add the lsn and
time of this abort to this message.(see function logicalrep_write_stream_abort)
When subscriber receives this message, it will update the replication origin.
(see function apply_handle_stream_abort and function RecordTransactionAbort)

2. Fixed missing settings for two GUCs (session_replication_role and
search_path) in apply background worker in patch 0001 and improved checking of
trigger functions in patch 0003.

Thanks to Hou Zhi Jie for adding the aborts message related infrastructure for
the first point.
Thanks to Shi Yu for pointing out the second point.

Attach the new patches.(only changed 0001 and 0003)

[1] - https://www.postgresql.org/message-id/OS3PR01MB6275FBD9359F8ED0EDE7E5459EDE9%40OS3PR01MB6275.jpnprd01.prod.outlook.com

Regards,
Wang wei

Attachment Content-Type Size
v9-0001-Perform-streaming-logical-transactions-by-backgro.patch application/octet-stream 93.3 KB
v9-0002-Test-streaming-apply-option-in-tap-test.patch application/octet-stream 68.9 KB
v9-0003-Add-some-checks-before-using-apply-background-wor.patch application/octet-stream 27.1 KB
v9-0004-Add-a-GUC-max_apply_bgworkers_per_subscription-to.patch application/octet-stream 6.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michail Nikolaev 2022-06-08 07:15:19 Re: CPU time for pg_stat_statement
Previous Message Jakub Wartak 2022-06-08 06:29:01 RE: effective_io_concurrency and NVMe devices