Re: Logical replication row filter loses unchanged toasted columns

On Sat, 15 Aug 2026 at 10:32, Shinya Kato <shinya11(dot)kato(at)gmail(dot)com> wrote:
>
> Thank you all for the discussion. We now have options A to E and
> Amit's Option 1 and 2, so let me sort the proposals by where each one
> intervenes.
>
> - Publication DDL time: forbid creating the publication (Matthias's
> D), or warn (Greg's E).
>
> - UPDATE time on the publisher: reject the UPDATE, per row in
> heap_update() (Amit's Option 1, Hou's 0001), or per statement in
> CheckCmdReplicaIdentity() (Amit's Option 2, Hou's 0002).
>
> - Decode time on the publisher: error in pgoutput_row_filter() (my A).
>
> - Apply time on the subscriber: error in apply_handle_insert() (Nikhil's patch).
>
> - Make it work instead of erroring: WAL-log the missing values, either
> always (my B) or for user-chosen columns (the INCLUDE-like idea
> upthread).
>
> - Document only (my C).
>
> The earlier a check runs, the more it prevents and the less it knows.
> The DDL and UPDATE time checks fire before anything is written to WAL,
> but they have to be conservative. Even Option 1 rejects an UPDATE
> whose old and new rows both match the filter, which replicates fine as
> a plain UPDATE today. The decode and apply time checks are precise,
> they fire exactly when a value is dropped, but by then the value is
> gone. Between those two, the apply time error is recoverable with
> ALTER SUBSCRIPTION SKIP while the decode time error leaves the slot
> stuck, so Nikhil's check supersedes my A.
>
> Given that, the combination I would aim for is:
>
> - All branches (15+): Nikhil's apply time error plus a documentation
> note. This is not redundant on master even after an UPDATE time check
> lands there, because a master subscriber can replicate from an older
> publisher that has no such check.
>
> - master: additionally reject at UPDATE time. I agree with Amit's lean
> towards Option 1. Option 2 rejects every UPDATE on a row-filtered
> table that merely has a toastable column outside the replica identity,
> which is close to D in impact.
>
> - Future: the INCLUDE-like logging in a separate thread, which would
> turn the remaining errors into working replication.
>
> I will review Nikhil's patch and Hou's patches later.
>
> Thoughts?

I don't understand the position of the LR developers here.

AFAIK, Logical Replication is (and has been) positioned as a
transparent add-on feature, that adds a new feature (replicating the
logical changes in a database) without removing functionality (such as
DML). LR's lack of support for certain functionalities (such as DDL,
or until recently sequences) didn't remove those functionalities from
the table or database that was configured for DDL, but instead the
feature was built in a way that replication couldn't be set up for
some features (sequences), or the replication stream would move into
an error state (most breaking DDL changes).

The solution that the developers seem to lean towards here is exactly
opposite to this: Enabling logical replication on a table breaks [^1]
(or would break [^2]) existing DML workloads.
To me, that looks like a clear inversion of responsibilities. I don't
think LR should push its hard problems to a user or workload that
might be unable to fix the relevant issues. If the logical
replication framework can't handle some configurations, it should
error out, the DDL handler should be adapted so that it can't be
configured in that way, or the framework should adapted to be able to
handle the configuration, but in no case should Logical Replication
push its limitations onto non-REPLICATION users that are just using
the table like any other normal table.

Kind regards,

Matthias van de Meent
Databricks (https://www.databricks.com)

[^1] tables without replication identity can recieve neither UPDATEs
nor DELETEs if those commands are included in the publication.
[^2] the general direction of this thread seems to lean towards
blocking DML on affected tables.