Re: Proposal: Conflict log history table for Logical Replication

From: Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>
To: shveta malik <shveta(dot)malik(at)gmail(dot)com>
Cc: Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Zhijie Hou <houzhijie22(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-10-22 11:53:18
Message-ID: CABdArM7POG_+2qj-hvXM9yDsgp8N2uCuFggdmHDJS2hrOh82KA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Oct 16, 2026 at 8:03 PM Nisha Moond <nisha(dot)moond412(at)gmail(dot)com> wrote:
>
> Attached is v82, which addresses the comments on v81:
>

We have not yet documented that, in rare cases, the subscriber can
still fail with the 1GB error when recording a conflict.

The 64kB cap on replica identity values is checked after the key
value's output function has produced its text, so it limits what we
store, not what we render. Normally that's fine: the publisher already
rendered the same value to send it, which keeps the text well within
the limit. That doesn't hold in a few rare cases:
- the key column has a different data type on the subscriber (e.g.
text published into a numeric[] column);
- binary = true, where the publisher never produced the text;
- the apply worker's bytea_output or quote_all_identifiers differs
from the publisher's.

In these cases a small stored value can render to more than 1GB, and
the apply worker errors out.

The attached top-up patch 0004 documents this limitation and adds an
XXX comment in build_index_key_json() describing these cases.
Patches 0001-0003 are unchanged.

--
Thanks,
Nisha

Attachment Content-Type Size
v82_1-0001-Implement-the-conflict-insertion-infrastructur.patch application/octet-stream 95.3 KB
v82_1-0002-Avoid-stalling-apply-on-the-conflict-log-table.patch application/octet-stream 25.4 KB
v82_1-0003-Address-review-comments-on-v81-0001.patch application/octet-stream 14.6 KB
v82_1-0004-Document-and-comment-that-replica-identity-ren.patch application/octet-stream 3.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Langote 2026-10-22 11:54:55 Re: PG19: two RI fast-path issues found while testing the batching revert
Previous Message Etsuro Fujita 2026-10-22 11:47:20 Re: Fix rescan costing of single-batch hash joins