Re: row filtering for logical replication

From: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
To: Ajin Cherian <itsajin(at)gmail(dot)com>, Euler Taveira <euler(at)eulerto(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Rahila Syed <rahilasyed90(at)gmail(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)enterprisedb(dot)com>, Önder Kalacı <onderkalaci(at)gmail(dot)com>, japin <japinli(at)hotmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, David Steele <david(at)pgmasters(dot)net>, Craig Ringer <craig(at)2ndquadrant(dot)com>, Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: row filtering for logical replication
Date: 2021-09-27 13:34:08
Message-ID: 2e1f3505-6533-f642-37bc-e75c15cfcde4@enterprisedb.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

I see no one responded to this important part of my review so far:

On 9/23/21 2:33 PM, Tomas Vondra wrote:
> 3) create_subscription.sgml
>
>     <literal>WHERE</literal> clauses, rows must satisfy all expressions
>     to be copied. If the subscriber is a
>
> I'm rather skeptical about the principle that all expressions have to
> match - I'd have expected exactly the opposite behavior, actually.
>
> I see a subscription as "a union of all publications". Imagine for
> example you have a data set for all customers, and you create a
> publication for different parts of the world, like
>
>   CREATE PUBLICATION customers_france
>      FOR TABLE customers WHERE (country = 'France');
>
>   CREATE PUBLICATION customers_germany
>      FOR TABLE customers WHERE (country = 'Germany');
>
>   CREATE PUBLICATION customers_usa
>      FOR TABLE customers WHERE (country = 'USA');
>
> and now you want to subscribe to multiple publications, because you want
> to replicate data for multiple countries (e.g. you want EU countries).
> But if you do
>
>   CREATE SUBSCRIPTION customers_eu
>          PUBLICATION customers_france, customers_germany;
>
> then you won't get anything, because each customer belongs to just a
> single country. Yes, I could create multiple individual subscriptions,
> one for each country, but that's inefficient and may have a different
> set of issues (e.g. keeping them in sync when a customer moves between
> countries).
>
> I might have missed something, but I haven't found any explanation why
> the requirement to satisfy all expressions is the right choice.
>
> IMHO this should be 'satisfies at least one expression' i.e. we should
> connect the expressions by OR, not AND.

Am I the only one finding the current behavior strange? What's the
reasoning supporting the current approach?

regards

--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2021-09-27 13:44:30 Re: Support for NSS as a libpq TLS backend
Previous Message Justin Pryzby 2021-09-27 13:32:44 Re: when the startup process doesn't (logging startup delays)