RE: Logical replication row filter loses unchanged toasted columns

From: "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Shinya Kato <shinya11(dot)kato(at)gmail(dot)com>
Cc: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Subject: RE: Logical replication row filter loses unchanged toasted columns
Date: 2026-08-31 03:15:19
Message-ID: TY4PR01MB177185C0B3D5613607629635194A92@TY4PR01MB17718.jpnprd01.prod.outlook.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, August 28, 2026 3:16 PM Shinya Kato <shinya11(dot)kato(at)gmail(dot)com> wrote:
> On Thu, Aug 27, 2026 at 11:23 AM Zhijie Hou (Fujitsu)
> <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> > Thanks for the comments. They look good to me and I have addressed them
> > in V4 patch.
>
> Thanks for working on this! I reviewed v4 and have two comments for now.

Thanks for the comments.

>
> 1. The extra logging does not actually depend on the key changing.
>
> ```
> + old_key_tuple = BuildOldKeyTuple(relation, &oldtup, newtup,
> + bms_overlap(modified_attrs, id_attrs) ||
> + id_has_external,
> + log_unchanged_external &&
> + HeapTupleHasExternal(&oldtup) &&
> + HeapTupleHasExternal(newtup),
> + &old_key_copied);
> ```
>
> The new columns are only kept when BuildOldKeyTuple() gets past its
> `if (!key_required) return NULL`, and the key_required passed above is
> `bms_overlap(modified_attrs, id_attrs) || id_has_external`, so
> id_has_external alone gets it there. HeapDetermineColumnsInfo() sets
> id_has_external when a replica identity column of the old tuple is
> stored externally, with no key change at all. So on a table whose
> replica identity covers a toasted column and which has another toasted
> column outside it, an UPDATE that touches neither cannot be
> transformed, yet still detoasts and flattens that column into the WAL
> record.

Right. I updated the patch to skip logging for this case.

>
> 2. The test never checks that val is actually stored out-of-line. If
> that ever changed, the test would keep passing without exercising the
> fix, since an inline value replicates fine anyway. Asserting
> pg_column_toast_chunk_id(val) IS NOT NULL on the publisher before the
> UPDATE would pin that down.

Added the test.

Here is the updated patch which addressed all comments including Kuroda-San's[1].

[1] https://www.postgresql.org/message-id/OS9PR01MB12149A8BE71B8C9E29A9C8CC1F5AD2%40OS9PR01MB12149.jpnprd01.prod.outlook.com

Best Regards,
Zhijie Hou

Attachment Content-Type Size
v5-0001-Log-unchanged-out-of-line-columns-for-row-filtere.patch application/octet-stream 22.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message shihao zhong 2026-08-31 03:48:45 [PATCH] pageinspect: validate line pointers before using them
Previous Message Zhijie Hou (Fujitsu) 2026-08-31 02:59:17 RE: Logical replication can lose an update after concurrent index invalidation