RE: Skipping logical replication transactions on subscriber side

From: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Greg Nancarrow <gregn4422(at)gmail(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, Alexey Lesovsky <lesovsky(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Skipping logical replication transactions on subscriber side
Date: 2022-03-14 09:50:41
Message-ID: OSZPR01MB6310F1C8F24C388DB7231525FD0F9@OSZPR01MB6310.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Mar 11, 2022 4:20 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> I've attached an updated version patch. This patch can be applied on
> top of the latest disable_on_error patch[1].
>

Thanks for your patch. Here are some comments for the v13 patch.

1. doc/src/sgml/ref/alter_subscription.sgml
+ Specifies the transaction's finish LSN of the remote transaction whose changes

Could it be simplified to "Specifies the finish LSN of the remote transaction
whose ...".

2.
I met a failed assertion, the backtrace is attached. This is caused by the
following code in maybe_start_skipping_changes().

+ /*
+ * It's a rare case; a past subskiplsn was left because the server
+ * crashed after preparing the transaction and before clearing the
+ * subskiplsn. We clear it without a warning message so as not confuse
+ * the user.
+ */
+ if (unlikely(MySubscription->skiplsn < lsn))
+ {
+ clear_subscription_skip_lsn(MySubscription->skiplsn, InvalidXLogRecPtr, 0,
+ false);
+ Assert(!IsTransactionState());
+ }

We want to clear subskiplsn in the case mentioned in comment. But if the next
transaction is a steaming transaction and this function is called by
apply_spooled_messages(), we are inside a transaction here. So, I think this
assertion is not suitable for streaming transaction. Thoughts?

3.
+ XLogRecPtr subskiplsn; /* All changes which committed at this LSN are
+ * skipped */

To be consistent, should the comment be changed to "All changes which finished
at this LSN are skipped"?

4.
+ After logical replication worker successfully skips the transaction or commits
+ non-empty transaction, the LSN (stored in
+ <structname>pg_subscription</structname>.<structfield>subskiplsn</structfield>)
+ is cleared.

Besides "commits non-empty transaction", subskiplsn would also be cleared in
some two-phase commit cases I think. Like prepare/commit/rollback a transaction,
even if it is an empty transaction. So, should we change it for these cases?

5.
+ * Clear subskiplsn of pg_subscription catalog with origin state update.

Should "with origin state update" modified to "with origin state updated"?

Regards,
Shi yu

Attachment Content-Type Size
backtrace.txt text/plain 1.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2022-03-14 09:53:44 Re: Column Filtering in Logical Replication
Previous Message Nitin Jadhav 2022-03-14 09:46:50 Re: Report checkpoint progress with pg_stat_progress_checkpoint (was: Report checkpoint progress in server logs)