Re: Skipping schema changes in publication

From: vignesh C <vignesh21(at)gmail(dot)com>
To: shveta malik <shveta(dot)malik(at)gmail(dot)com>
Cc: "David G(dot) Johnston" <david(dot)g(dot)johnston(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Shlok Kyal <shlok(dot)kyal(dot)oss(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>, YeXiu <1518981153(at)qq(dot)com>, Ian Lawrence Barwick <barwick(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Skipping schema changes in publication
Date: 2026-02-10 17:41:03
Message-ID: CALDaNm2EU7qdEnGmirzVn9a1O-6y3q5c7nzsBZCuyVjaPS37mg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, 10 Feb 2026 at 11:35, shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> On Mon, Feb 9, 2026 at 11:52 AM David G. Johnston
> <david(dot)g(dot)johnston(at)gmail(dot)com> wrote:
> >
> > On Sunday, February 8, 2026, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >>
> >> On Mon, Feb 9, 2026 at 6:41 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> >>
> >> > Hi Amit.
> >> >
> >> > I understand there can be some tricky scenarios where partitions are
> >> > involved, but I was not sure why "pub1: FOR ALL Tables EXCEPT (tab1)
> >> > and pub2: FOR TABLE tab1" is an example of contradictory behaviour.
> >> >
> >> > Consider if the publisher has 3 tables tab1,tab2,tab3:
> >> > Here, "pub1: FOR ALL Tables EXCEPT (tab1)" is like a shorthand for
> >> > saying "pub1: FOR TABLE tab2,tab3"
> >> > So what's wrong for the subscriber to combine pub1 and pub2 in this case?
> >> >
> >>
> >> It is because one of the publications (pub2) indicates to include a
> >> particular table tab1 and the other one (pub1) to exclude the same
> >> table. And things become much more complex when the Except list
> >> contains partitions as shown in Shveta's example. So, I think it makes
> >> sense to keep things simple at least for the first version, we can
> >> consider to uplift this restriction if we see some use cases from the
> >> field.
> >>
> >>
> >
> > I’m with Peter here - I do not think it is wise to expose the exception listing outside the publication. Publication combinations should be purely additive in much the same way grants are in the system. Except lists are internal shorthand for describing the positive list of tables a publication makes available - all tables except.
> >
>
> The earlier case -
> pub1: FOR ALL TABLES EXCEPT (tab1)
> pub2: FOR TABLE tab1 WHERE (c = 99)
>
> seems a valid scenario, and we are currently evaluating its
> implementation feasibility under Approach 1.
>
> OTOH, subscribing to two different publications that are both defined
> as 'FOR ALL TABLES' but have different EXCEPT lists introduces
> unnecessary implementation complexity without a clear business use
> case. This becomes especially complex when the publications exclude
> different partitions of the same partitioned table. For example:
>
> pub1: FOR ALL TABLES EXCEPT (part1, part2) WITH
> (publish_via_partition_root=true)
> pub2: FOR ALL TABLES EXCEPT (part7) WITH
> (publish_via_partition_root=false)
>
> IMO, there is no clear need for a user to create multiple 'ALL TABLES'
> publications with different EXCEPT lists and then combine them at the
> subscriber level. Given this, to keep the patch simpler, we plan to
> emit an error for this scenario (multi-pub EXCEPTs case) for now. If a
> valid requirement emerges in the future, we can revisit and consider
> supporting it.

I agree with this. This has been handled.
Additionally it also resolves conflicts between overlapping
publications with differing publish_via_partition_root settings. When
a partition is excluded from a "via root" publication but included in
another table publication with "via root" as false, then it is
published through its root ancestor by default to give priority to the
root table.
Ex:
-- Publisher tables
CREATE TABLE tab_root (id int, range_col int) PARTITION BY RANGE (range_col);
CREATE TABLE tab_part_1 PARTITION OF tab_root FOR VALUES FROM (1) TO
(1000) PARTITION BY RANGE (range_col);
CREATE TABLE tab_part_2 PARTITION OF tab_root FOR VALUES FROM (1000)
TO (2000) PARTITION BY RANGE (range_col);
CREATE TABLE tab_part_1_p1 PARTITION OF tab_part_1 FOR VALUES FROM (1) TO (500);
CREATE TABLE tab_part_1_p2 PARTITION OF tab_part_1 FOR VALUES FROM
(500) TO (1000);
CREATE TABLE tab_part_2_p1 PARTITION OF tab_part_2 FOR VALUES FROM
(1000) TO (1500);
CREATE TABLE tab_part_2_p2 PARTITION OF tab_part_2 FOR VALUES FROM
(1500) TO (2000);

-- Publication
CREATE PUBLICATION pub1 for ALL TABLES EXCEPT table (tab_part_1_p1,
tab_part_1_p2) WITH (PUBLISH_VIA_PARTITION_ROOT=true);
CREATE PUBLICATION pub2 for table tab_part_1_p2 WITH
(PUBLISH_VIA_PARTITION_ROOT=false);

-- Subscriber tables
CREATE TABLE tab_root (id int, range_col int);
CREATE TABLE tab_part_1 (id int, range_col int);
CREATE TABLE tab_part_2 (id int, range_col int);
CREATE TABLE tab_part_1_p1 (id int, range_col int);
CREATE TABLE tab_part_1_p2 (id int, range_col int);
CREATE TABLE tab_part_2_p1 (id int, range_col int);
CREATE TABLE tab_part_2_p2 (id int, range_col int);

Subscription with both pub1 and pub2 publications.:
create subscription sub1 connection 'dbname=postgres host=localhost
port=5432' publication pub1,pub2;

Consider the insert statements:
insert into tab_part_1_p1 values(1,100);
insert into tab_part_1_p2 values(2,600);
insert into tab_part_2_p1 values(3,1100);
insert into tab_part_2_p2 values(4,1600);

The tab_part_2_p2 table's row is replicated to the subscriber’s
tab_root table, since root table publications take precedence.
postgres=# select * from tab_root ;
id | range_col
----+-----------
2 | 600
3 | 1100
4 | 1600
(3 rows)

The attached v42 version patch has the changes for the same.

Regards,
Vignesh

Attachment Content-Type Size
v42-0001-Skip-publishing-the-tables-specified-in-EXCEPT-T.patch application/octet-stream 102.0 KB
v42-0002-Extended-tests-for-EXCEPT-TABLE-patch.patch application/octet-stream 4.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2026-02-10 17:58:23 Re: Instability in postgres_fdw regression tests
Previous Message Pierre Ducroquet 2026-02-10 17:39:40 llvmjit - improve code generated in O0