RE: Replica Identity check of partition table on subscriber

From: "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
Subject: RE: Replica Identity check of partition table on subscriber
Date: 2022-06-10 08:45:04
Message-ID: OSZPR01MB63107D0831086AF309CEFFEBFDA69@OSZPR01MB6310.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, June 9, 2022 7:02 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> > I think one approach to fix it is to check the target partition in this case,
> > instead of the partitioned table.
> >
>
> This approach sounds reasonable to me. One minor point:
> +/*
> + * Check that replica identity matches.
> + *
> + * We allow for stricter replica identity (fewer columns) on subscriber as
> + * that will not stop us from finding unique tuple. IE, if publisher has
> + * identity (id,timestamp) and subscriber just (id) this will not be a
> + * problem, but in the opposite scenario it will.
> + *
> + * Don't throw any error here just mark the relation entry as not updatable,
> + * as replica identity is only for updates and deletes but inserts can be
> + * replicated even without it.
> + */
> +static void
> +logicalrep_check_updatable(LogicalRepRelMapEntry *entry)
>
> Can we name this function as logicalrep_rel_mark_updatable as we are
> doing that? If so, change the comments as well.
>

OK. Modified.

> > When trying to fix it, I saw some other problems about updating partition
> map
> > cache.
> >
> > a) In logicalrep_partmap_invalidate_cb(), the type of the entry in
> > LogicalRepPartMap should be LogicalRepPartMapEntry, instead of
> > LogicalRepRelMapEntry.
> >
> > b) In logicalrep_partition_open(), it didn't check if the entry is valid.
> >
> > c) When the publisher send new relation mapping, only relation map cache
> will be
> > updated, and partition map cache wouldn't. I think it also should be updated
> > because it has remote relation information, too.
> >
>
> Is there any test case that can show the problem due to your above
> observations?
>

Please see the following case.

-- publisher
create table tbl (a int primary key, b int);
create publication pub for table tbl;

-- subscriber
create table tbl (a int primary key, b int, c int) partition by range (a);
create table child partition of tbl default;

-- publisher, make cache
insert into tbl values (1,1);
update tbl set a=a+1;
alter table tbl add column c int;
update tbl set c=1 where a=2;

-- subscriber
postgres=# select * from tbl;
a | b | c
---+---+---
2 | 1 |
(1 row)

The value of column c updated failed on subscriber.
And after applying the first patch, it would work fine.

I have added this case to the first patch. Also add a test case for the second
patch.

Attach the new patches.

Regards,
Shi yu

Attachment Content-Type Size
v2-0001-Fix-partition-map-cache-issues.patch application/octet-stream 9.1 KB
v2-0002-Check-partition-table-replica-identity-on-subscri.patch application/octet-stream 8.7 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jakub Wartak 2022-06-10 08:52:49 RE: pgcon unconference / impact of block size on performance
Previous Message Peter Smith 2022-06-10 08:39:04 Re: Handle infinite recursion in logical replication setup