Re: Proposal: Conflict log history table for Logical Replication

On Sat, May 30, 2026 at 3:24 AM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Fri, May 29, 2026 at 5:11 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Thu, May 21, 2026 at 9:51 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> > >
> > > On Wed, May 20, 2026 at 3:05 PM vignesh C <vignesh21(at)gmail(dot)com> wrote:
> > > >
> > > > Rest of the comments were fixed.
> > > > The attached v37 version patch has the changes for the same. Also
> > > > Peter's comments on the documentation patch from [1] and Shveta's
> > > > comments from [2] are addressed in the attached patch.
> > > >
> > >
> > > Here are few comments based on v37 testing:
> > >
> > > 1) Should we consider using TOAST tables for tuple-data columns like
> > > remote_tuple and local_conflicts (the JSON columns)?
> > > This may be a corner case, but if the tuple data becomes too large to
> > > fit into an 8KB heap tuple, then the apply worker keeps failing while
> > > inserting into the CLT with errors like:
> > >
> > > ERROR: row is too big: size 19496, maximum size 8160
> > > LOG: background worker "logical replication apply worker" (PID
> > > 41226) exited with exit code 1
> > >
> >
> > In the docs, it is mentioned: "column_value is the column value. The
> > large column values are truncated to 64 bytes." [1], so I wonder, if
> > we follow this why we need toast entries? Did you tried any case where
> > you are getting above ERROR?
>
> But in this case we are talking about the JSON column of the CLT which
> might contain a full local tuple or even multiple local tuples if a
> remote tuple conflicts with multiple local rows. So, IMHO, we need a
> toast table. Nisha, have you already tested the scenario? If yes, can
> you share your test case?
>

Hi Dilip, Amit,
Yes, I tested the scenario. Used below steps to reproduce the error:

#Publisher:
CREATE TABLE fat2 (id int PRIMARY KEY, col1 text, col2 text);
INSERT INTO fat2 VALUES (
1,
(SELECT string_agg(md5(i::text), '') FROM generate_series(1, 200) i),
(SELECT string_agg(md5(i::text), '') FROM generate_series(201, 400) i)
);
ALTER TABLE fat2 REPLICA IDENTITY FULL;
CREATE PUBLICATION p3 FOR TABLE fat2;

#Subscriber
-- create subscription s3 for publication p3 with conflict log table
(after table syncs):
-- modifying the row locally
UPDATE fat2 SET col1 = (SELECT string_agg(md5(i::text), '') FROM
generate_series(601, 800) i) WHERE id = 1;

#Publisher (triggers the conflict):
UPDATE fat2 SET col1 = (SELECT string_agg(md5(i::text), '') FROM
generate_series(801, 1000) i) WHERE id = 1;

Above should cause the reported failure.

--
Thanks,
Nisha