Re: Proposal: Conflict log history table for Logical Replication

On Tue, Oct 13, 2026 at 12:31 PM Peter Smith <smithpb2250(at)gmail(dot)com> wrote:
>
> Shveta pointed me to this post explaining about why the oversized
> columns cannot simply be truncated.
>
> I was wondering if a middle-ground solution would be useful. One where
> the data is still flagged as truncated has the advantages of Amit's
> proposal in that it cannot be confused with byte-exact data, but it
> still carries some truncated value which might (?) be helpful to a
> human reader.
>
> e.g.
>
> Option1 - This option is no good because JSON is too large
> "b" : "some really long value that it too long and would exceed limits"
>
> Option 2 - This option is no good because SQL cannot tell if it is
> really truncated or not.
> "b" : "some really long value that ..."
>
> Option 3. This option is what Ami proposed in the above mail.
> "b" : {"omitted": true, "length": 190000000}
>
> Option 4 - This option is the middle-ground that I was wondering about.
> "b" : {"truncated": true, "length": 190000000, "prefix": "some really
> long value that ..." }
>

Hi Peter,

Thanks for the suggestions. I looked at this same option (Option 4) a
couple of weeks ago in [1] and argued against it then, but that
reasoning doesn't hold since v80.

My concern was that building a prefix for arrays, composites or jsonb
etc. means rendering the whole value first, which could itself go past
1GB. But we no longer check the stored size up front. The output
function is always called, and only the length of its result is
compared with the cap. So the text is already in hand when we write
the marker, and keeping its first few bytes costs almost nothing and
adds no new path to the 1GB limit error. So it might be worth
discussing again.

However, if we bring back the stored-size pre-check that was suggested
as an optimization[2], oversized values would be skipped without
rendering, and a prefix would again need a full render. In that case,
IMO, we should not go in this direction.

I'm not sure what length would be useful. For text-like keys even ~100
bytes would help. For large arrays or nested composites, the first
bytes are mostly structure like ("(""(""((..., so the prefix wouldn't
tell the user much.

Thoughts?

[1] https://www.postgresql.org/message-id/CABdArM6_FvTMr2Pz-n5Af9bxXBSHjsCVn1931XTCK%3D6sMHQ3JA%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAFiTN-uxsFDUGfuf1e6D1dPGfJYv2Gm_dOkVYohKuX-Uxm3_aA%40mail.gmail.com

--
Thanks,
Nisha