Re: row filtering for logical replication

On Wed, Sep 8, 2021 at 7:59 PM Ajin Cherian <itsajin(at)gmail(dot)com> wrote:
>
> On Wed, Sep 1, 2021 at 9:23 PM Euler Taveira <euler(at)eulerto(dot)com> wrote:
> >

> Somehow this approach of either new_tuple or old_tuple doesn't seem to
> be very fruitful if the user requires that his replica is up-to-date
> based on the filter condition. For that, I think you will need to
> convert UPDATES to either INSERTS or DELETES if only new_tuple or
> old_tuple matches the filter condition but not both matches the filter
> condition.
>
> UPDATE
> old-row (match) new-row (no match) -> DELETE
> old-row (no match) new row (match) -> INSERT
> old-row (match) new row (match) -> UPDATE
> old-row (no match) new-row (no match) -> (drop change)
>

Adding a patch that strives to do the logic that I described above.
For updates, the row filter is applied on both old_tuple
and new_tuple. This patch assumes that the row filter only uses
columns that are part of the REPLICA IDENTITY. (the current patch-set
only
restricts this for row-filters that are delete only)
The old_tuple only has columns that are part of the old_tuple and have
been changed, which is a problem while applying the row-filter. Since
unchanged REPLICA IDENTITY columns
are not present in the old_tuple, this patch creates a temporary
old_tuple by getting such column values from the new_tuple and then
applies the filter on this hand-created temp old_tuple. The way the
old_tuple is created can be better optimised in future versions.

This patch also handles the problem reported by Houz in [1]. The patch
assumes a fix proposed by Dilip in [2]. This is the case
where toasted unchanged RI columns are not detoasted in the new_tuple
and has to be retrieved from disk during decoding. Dilip's fix
involved updating the detoasted value in the old_tuple when writing to
WAL. In the problem reported by Hou, when the row filter
is applied on the new_tuple and the decoder attempts to detoast the
value in the new_tuple and if the table was deleted at that time, the
decode fails.
To avoid this, in such a situation, the untoasted value in the
old_tuple (fix by Dilip) is copied to the new_tuple before the
row_filter is applied.
I have also refactored the way Peter initializes the row_filter by
moving it into a separate function before the insert/update/delete
specific logic is applied.

I have not changed any of the first 5 patches, just added my patch 006
at the end. Do let me know of any comments on this approach.

[1] - https://www.postgresql.org/message-id/OS0PR01MB571618736E7E79309A723BBE94E99%40OS0PR01MB5716.jpnprd01.prod.outlook.com
[2] - https://www.postgresql.org/message-id/OS0PR01MB611342D0A92D4F4BF26C0F47FB229@OS0PR01MB6113.jpnprd01.prod.outlook.com
regards,
Ajin Cherian
Fujitsu Australia