Re: Proposal: Conflict log history table for Logical Replication

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zhijie Hou <houzhijie22(at)gmail(dot)com>, shveta malik <shvetamalik(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-10-26 09:14:23
Message-ID: CAJpy0uBfBx3fbZ=ySEsvSgNQ4k3=00wGqLUv44Z6_Y4396Y3sw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Oct 22, 2026 at 5:23 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
>
> On Fri, Oct 16, 2026 at 8:03 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
> >
> > Attached is v82, which addresses the comments on v81:
> >
>
> We have not yet documented that, in rare cases, the subscriber can
> still fail with the 1GB error when recording a conflict.
>
> The 64kB cap on replica identity values is checked after the key
> value's output function has produced its text, so it limits what we
> store, not what we render. Normally that's fine: the publisher already
> rendered the same value to send it, which keeps the text well within
> the limit. That doesn't hold in a few rare cases:
> - the key column has a different data type on the subscriber (e.g.
> text published into a numeric[] column);
> - binary = true, where the publisher never produced the text;
> - the apply worker's bytea_output or quote_all_identifiers differs
> from the publisher's.
>
> In these cases a small stored value can render to more than 1GB, and
> the apply worker errors out.
>
> The attached top-up patch 0004 documents this limitation and adds an
> XXX comment in build_index_key_json() describing these cases.
> Patches 0001-0003 are unchanged.

I agree with these scenarios and the need for documentation. But I
have polished the doc change in patch 0004 a bit and attached the
updated patch as text. Please take it if you agree. If we prefer not
to elaborate on the details, we can drop the scenarios while retaining
the rest of the wording.

thanks
Shveta

Attachment Content-Type Size
0001-Document-and-comment-that-replica-identity-rendering.patch.txt text/plain 4.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2026-10-26 09:23:56 Re: Re: [PATCH] Fix segmentation fault caused by reentrancy in RI_Fkey_cascade_del (ri_triggers.c)
Previous Message Tender Wang 2026-10-26 09:05:03 Re: Self-join elimination discards differing row lock wait policies