Re: Proposal: Conflict log history table for Logical Replication

On Tue, Oct 13, 2026 at 7:05 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
>
> On Tue, Oct 13, 2026 at 12:31 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
> >
> > Shveta pointed me to this post explaining about why the oversized
> > columns cannot simply be truncated.
> >
> > I was wondering if a middle-ground solution would be useful. One where
> > the data is still flagged as truncated has the advantages of Amit's
> > proposal in that it cannot be confused with byte-exact data, but it
> > still carries some truncated value which might (?) be helpful to a
> > human reader.
> >
> > e.g.
> >
> > Option1 - This option is no good because JSON is too large
> > "b" : "some really long value that it too long and would exceed limits"
> >
> > Option 2 - This option is no good because SQL cannot tell if it is
> > really truncated or not.
> > "b" : "some really long value that ..."
> >
> > Option 3. This option is what Ami proposed in the above mail.
> > "b" : {"omitted": true, "length": 190000000}
> >
> > Option 4 - This option is the middle-ground that I was wondering about.
> > "b" : {"truncated": true, "length": 190000000, "prefix": "some really
> > long value that ..." }
> >
>
> Hi Peter,
>
> Thanks for the suggestions. I looked at this same option (Option 4) a
> couple of weeks ago in [1] and argued against it then, but that
> reasoning doesn't hold since v80.
>
> My concern was that building a prefix for arrays, composites or jsonb
> etc. means rendering the whole value first, which could itself go past
> 1GB. But we no longer check the stored size up front. The output
> function is always called, and only the length of its result is
> compared with the cap. So the text is already in hand when we write
> the marker, and keeping its first few bytes costs almost nothing and
> adds no new path to the 1GB limit error. So it might be worth
> discussing again.
>
> However, if we bring back the stored-size pre-check that was suggested
> as an optimization[2], oversized values would be skipped without
> rendering, and a prefix would again need a full render. In that case,
> IMO, we should not go in this direction.
>

Thanks for the info.

> I'm not sure what length would be useful. For text-like keys even ~100
> bytes would help. For large arrays or nested composites, the first
> bytes are mostly structure like ("(""(""((..., so the prefix wouldn't
> tell the user much.

Yes. I understood non text fields may not be readable. Perhaps the
"prefix" is optional only and shown only for text fields?

>
> Thoughts?

I was not aware this option was already previously discussed.and
dismissed so I am happy to drop the suggestion. OTOH, since this same
question has been raised separately a couple of times [1], maybe it
warrants a comment in the code to explain that the prefix is not
stored deliberately and the reason why.

======
[1] https://www.postgresql.org/message-id/CAJZSXGpp46n6VOnEsqod9b3dz%2BZ45dxoqcPcyEWTA9T5WYf9tw%40mail.gmail.com

Kind Regards,
Peter Smith.
Fujitsu Australia