Re: Skipping logical replication transactions on subscriber side

From: Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
Cc: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Greg Nancarrow <gregn4422(at)gmail(dot)com>, "tanghy(dot)fnst(at)fujitsu(dot)com" <tanghy(dot)fnst(at)fujitsu(dot)com>, "osumi(dot)takamichi(at)fujitsu(dot)com" <osumi(dot)takamichi(at)fujitsu(dot)com>, Alexey Lesovsky <lesovsky(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Skipping logical replication transactions on subscriber side
Date: 2021-12-02 15:08:02
Message-ID: f716f584-65d0-fe83-2e84-53426631739a@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


On 02.12.21 07:48, Amit Kapila wrote:
> a. ALTER SUBSCRIPTION ... [SET|RESET] SKIP TRANSACTION xxx;
> b. Alter Subscription <sub_name> SET ( subscription_parameter [=value]
> [, ... ] );
> c. Alter Subscription <sub_name> On Error ( subscription_parameter
> [=value] [, ... ] );
> d. Alter Subscription <sub_name> SKIP ( subscription_parameter
> [=value] [, ... ] );
> where subscription_parameter can be one of:
> xid = <xid_val>
> lsn = <lsn_val>
> ...

> As per discussion till now, option (d) seems preferable.

I agree.

> In this, we
> need to see how and what to allow as options. The simplest way for the
> first version is to just allow one xid to be specified at a time which
> would mean that specifying multiple xids should error out. We can also
> additionally allow specifying operations like 'insert', 'update',
> etc., and then relation list (list of oids). What that would mean is
> that for a transaction we can allow which particular operations and
> relations we want to skip.

I don't know how difficult it would be, but allowing multiple xids might
be desirable. But this syntax gives you flexibility, so we can also
start with a simple implementation.

> I am not sure what exactly we can provide to users to allow skipping
> initial table sync as we can't specify XID there. One option that
> comes to mind is to allow specifying a combination of copy_data and
> relid to skip table sync for a particular relation. We might think of
> not doing anything for table sync workers but not sure if that is a
> good option.

I don't think this feature should affect tablesync. The semantics are
not clear, and it's not really needed. If the tablesync doesn't work,
you can try the setup again from scratch.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2021-12-02 15:44:02 Re: Can I assume relation would not be invalid during from ExecutorRun to ExecutorEnd
Previous Message Peter Eisentraut 2021-12-02 15:04:33 Re: Is ssl_crl_file "SSL server cert revocation list"?