Re: Logical replication row filter loses unchanged toasted columns

On Fri, Aug 14, 2026 at 4:50 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Thu, Aug 13, 2026 at 7:20 PM Matthias van de Meent
> <boekewurm+postgres(at)gmail(dot)com> wrote:
> >
> > On Wed, 12 Aug 2026 at 05:23, Shinya Kato <shinya11(dot)kato(at)gmail(dot)com> wrote:
> > >
> > > I see three ways to deal with this.
> > >
> > > Option A: detect the missing value in pgoutput_row_filter() and raise
> > > an error naming the table and the column, trading silent data loss for
> > > a loud failure. [...]
> > >
> > > Option B: when a table belongs to a publication with a row filter,
> > > make heap_update() log the whole old tuple, as it already does for
> > > REPLICA IDENTITY FULL. [...]
> > >
> > > Option C: document the restriction and leave the behavior alone. [...]
> >
> > Or, an option D: Forbid the creation (and use) of filtered publication
> > table definitions for tables which contain a non-identity
> > varlena-typed column (i.e. the type's typlen is -1).
> >
>
> I think even if we want to block operations that can create such a
> situation, we should reject only the specific updates that lead to the
> problem, not every update on a table that merely has the potential for
> it. We already do something similar: UPDATE/DELETE is rejected when
> there's no replica identity and the table's publications publish those
> operations. I'd like to apply the same principle here.
>
> With that in mind, I could think of following two options:
>
> Option 1
> Check at DML time, inside heap_update(): Detect the problem per-row,
> at the point where old/new tuple data is actually available when
> following conditions are met: the relation is published and has
> UPDATEs enabled, (b) some publication defines a row filter on it, (c)
> the replica identity key changed value in this UPDATE, (d) the old
> tuple has some externally-stored (TOASTed) attribute
> (HeapTupleHasExternal()), and (e) some specific non-replica-identity
> column's value is unchanged and still stored out-of-line.
>
> Only an UPDATE that actually satisfies all five conditions is
> rejected, with an error naming the offending column. All other UPDATEs
> on the same table proceed normally, including ones that don't touch
> the key, or ones where the TOASTed column did change.

It looks like this bug can happen when all of the above five
conditions are met, which seems to be narrow in practice. If the
affected cases are that narrow, why don't we log the whole old tuple
image in exactly that case, instead of erroring out? While it does
write more WAL, but only in that narrow case, It would be better than
requiring users to change RI setting or publication settings. Also, I
guess it can be back-patched as it neither adds a new WAL record type
nor changes the existing WAL format. Ideally, it would be sufficient
to write RI + unchanged out-of-line column data. But that needs new
logic to select the attribute, so I would leave it for the master.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com