Re: Proposal: Conflict log history table for Logical Replication

On Wed, Oct 14, 2026 at 3:20 AM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> On Tue, Oct 13, 2026 at 7:05 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> >
> > On Tue, Oct 13, 2026 at 12:31 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> > >
> > > Shveta pointed me to this post explaining about why the oversized
> > > columns cannot simply be truncated.
> > >
> > > I was wondering if a middle-ground solution would be useful. One where
> > > the data is still flagged as truncated has the advantages of Amit's
> > > proposal in that it cannot be confused with byte-exact data, but it
> > > still carries some truncated value which might (?) be helpful to a
> > > human reader.
> > >
> > > e.g.
> > >
> > > Option1 - This option is no good because JSON is too large
> > > "b" : "some really long value that it too long and would exceed limits"
> > >
> > > Option 2 - This option is no good because SQL cannot tell if it is
> > > really truncated or not.
> > > "b" : "some really long value that ..."
> > >
> > > Option 3. This option is what Ami proposed in the above mail.
> > > "b" : {"omitted": true, "length": 190000000}
> > >
> > > Option 4 - This option is the middle-ground that I was wondering about.
> > > "b" : {"truncated": true, "length": 190000000, "prefix": "some really
> > > long value that ..." }
> > >
> >
> > Hi Peter,
> >
> > Thanks for the suggestions. I looked at this same option (Option 4) a
> > couple of weeks ago in [1] and argued against it then, but that
> > reasoning doesn't hold since v80.
> >
> > My concern was that building a prefix for arrays, composites or jsonb
> > etc. means rendering the whole value first, which could itself go past
> > 1GB. But we no longer check the stored size up front. The output
> > function is always called, and only the length of its result is
> > compared with the cap. So the text is already in hand when we write
> > the marker, and keeping its first few bytes costs almost nothing and
> > adds no new path to the 1GB limit error. So it might be worth
> > discussing again.
> >
> > However, if we bring back the stored-size pre-check that was suggested
> > as an optimization[2], oversized values would be skipped without
> > rendering, and a prefix would again need a full render. In that case,
> > IMO, we should not go in this direction.
> >
>
> Thanks for the info.
>
> > I'm not sure what length would be useful. For text-like keys even ~100
> > bytes would help. For large arrays or nested composites, the first
> > bytes are mostly structure like ("(""(""((..., so the prefix wouldn't
> > tell the user much.
>
> Yes. I understood non text fields may not be readable. Perhaps the
> "prefix" is optional only and shown only for text fields?
>
> >
> > Thoughts?
>
> I was not aware this option was already previously discussed.and
> dismissed so I am happy to drop the suggestion. OTOH, since this same
> question has been raised separately a couple of times [1], maybe it
> warrants a comment in the code to explain that the prefix is not
> stored deliberately and the reason why.
>

Okay. I added a comment at the marker code saying the prefix is left
out deliberately.

--
Thanks,
Nisha