Re: Logical replication row filter loses unchanged toasted columns

On Thu, 13 Aug 2026 at 17:19, Greg Sabino Mullane <htamfids(at)gmail(dot)com> wrote:
>
> On Thu, Aug 13, 2026 at 9:50 AM Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com> wrote:
>>
>> Or, an option D: Forbid the creation (and use) of filtered publication table definitions for tables which contain a non-identity varlena-typed column (i.e. the type's typlen is -1).
>
>
> I think it's too late for that: option A seems better until we get a proper fix.

What do you mean by "too late for that"?

If you mean "there are already systems with publications with filters
on tables with published varlena non-identity columns" then you're
right that those systems exist, but that shouldn't preclude us from
starting to raise errors when the user wants to create a new (or start
using an existing) publication that we can assume to be broken; We've
disabled and removed inherently broken features before, why shouldn't
we do that here?

I don't think there is a more proper fix than this option D.
Publications are downstream of the LR decoder (plugins can use
publications, but , and including publication information in the
decision-making process upstream of that (such as, in tableam's update
handler) we would have to break through several layers of
abstractions. And, after all of that, it'd still leave decoders which
don't use the publication tables with broken data, because those don't
necessarily track the data they export through pg_publication et al.

>> we should not allow users to create publications of which we know ahead of time that the
>> data stream is likely to break on our side.
>
>
> Maybe instead* we issue a warning on creation (again, until we get a real fix, which I think is doable and probably needed for more than just this use case)

I don't see how we can get a "real" fix. Detoasting and WAL-logging
all external columns "because a logical plugin may need to see this
column if it has filtering" would explode the amount of WAL used in
updates; it'd be indistinguishable from REPLICATION IDENTITY FULL.

> * To be clear, I'm recommending the warning (call it option E) in addition to option A

The warning would have to say something along the lines of "Hey, your
table has varlena columns and a filter, be aware your column data may
be lost. Oh, don't worry, we'll detect it when we lost your data and
stop the replication stream.", and if the message says something along
those lines then we'd better just plainly disallow such publications
by throwing errors, so that we don't leave a known and documented way
to lose data or halt publications when the user is using PG normally
in all possible ways. "Sorry, TOAST happened" is not a good argument
to structurally halt replication slots.

Kind regards,

Matthias van de Meent
Databricks (https://www.databricks.com)