Re: Column Filtering in Logical Replication

Hi,

Here's an updated version of the patch, rebased to current master. Parts
0002 and 0003 include various improvements based on review by me and
another one by Peter Smith [1].

Part 0003 reworks and significantly extends the TAP test, to exercise
various cases related to changes of replica identity etc. discussed in
this thread. Some of the tests however still fail, because the behavior
was not updated - I'll work on that once we agree what the expected
behavior is.

1) partitioning with pubviaroot=true

The main set of failures is related to partitions with different replica
identities and (pubviaroot=true), some of which may be mismatching the
column list. There are multiple such test cases, depending on how the
inconsistency is introduced - it may be there from the beginning, the
column filter may be modified after adding the partitioned table to the
publication, etc.

I think the expected behavior is to prohibit such cases from happening,
by cross-checking the column filter when adding the partitioned table to
publication, attaching a partition or changing a column filter.

2) merging multiple column filters

When the table has multiple column filters (in different publications),
we need to merge them. Which works, except that FOR ALL TABLES [IN
SCHEMA] needs to be handled as "has no column filter" (and replicates
everything).

3) partitioning with pubivaroot=false

When a partitioned table is added with (pubviaroot=false), it should not
be subject to column filter on the parent relation, which is the same
behavior used by the row filtering patch.

regards

[1]
https://www.postgresql.org/message-id/CAHut%2BPtc7Rh187eQKrxdUmUNWyfxz7OkhYAX%3DAW411Qwxya0LQ%40mail.gmail.com

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company