Re: Proposal: Conflict log history table for Logical Replication

From: Zhijie Hou <houzhijie22(at)gmail(dot)com>
To: shveta malik <shveta(dot)malik(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Dilip Kumar <DilipBalaut(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, shveta malik <shvetamalik(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-10-26 15:24:30
Message-ID: CAFvd2n8-M5G9kktzmaSFYnjXHHkJt019ZVHtKONxKsDvfOB7vA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

On Mon, Oct 26, 2026 at 12:59 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> On Mon, Oct 26, 2026 at 9:52 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Mon, Oct 26, 2026 at 9:08 AM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
> > >
> > > On Sat, Oct 24, 2026 at 11:42 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > > >
> > > > The current use of the "datestyle" and other GUCs in libpqwalreceiver
> > > > or in other places is to keep the format portable across nodes but
> > > > here the CLT output will be usually human-readable. So, the same
> > > > analogy doesn't follow here. I am not convinced that this required
> > > > mid-way changing the GUCs temporarily. I feel we can document this
> > > > column and if we still get a request from field for portability of
> > > > this field(s) then we can consider using GUCs to make the format
> > > > consistent.
> > > >
> > >
> > > But it causes ambiguous results as shown in [1]. I am not sure how
> > > documentation will help here. As I see it, the stored key must
> > > convert back to the original value, not to some other value,
> > > especially since the patch’s own docs recommend casting it back to the
> > > column type before comparing. And I don’t think libpqwalreceiver does
> > > this for portability: its comment says it forces these GUCs so values
> > > are output “in a form that is unambiguous to the subscriber.” IMO,
> > > that’s the same goal here: avoiding a wrong stored value, not
> > > portability.
> > >
> >
> > That is a cross-node compatibility case and similarly pg_dump's output
> > could be restored on another node. I feel if a user changes the
> > datestyle or similar settings in-between different conflict-insertions
> > the results can change accordingly. We shouldn't take such guarantees.
> > It should be documented somewhere along with the following text [1],
> > we can probably give a small example as well.
> >
> > [1]
> > + <para>
> > + The apply worker's <xref linkend="guc-bytea-output"/> and
> > + <xref linkend="guc-quote-all-identifiers"/> settings change how a value
> > + is written without changing the value itself. If these settings differ
> > + between conflicts, the same key can be recorded in different but
> > + equivalent forms, and comparing the recorded text may miss some of them.
>
> Okay. Let's do this, we can decide further if we have any real
> customer's feedback.

Here is the updated patch set based on the discussion. The GUC-related logic
has been removed and doc is added. I've also improved a few comments about the
oversized column value and the reason for referring to the table's column name
rather than the index's.

Only 0003 is changed.

--
Best Regards,
Zhijie Hou

Attachment Content-Type Size
v84-0004-Document-and-comment-that-replica-identity-rende.patch application/octet-stream 3.3 KB
v84-0002-Avoid-stalling-apply-on-the-conflict-log-table-w.patch application/octet-stream 27.3 KB
v84-0003-Address-review-comments-on-v81-0001.patch application/octet-stream 10.4 KB
v84-0001-Implement-the-conflict-insertion-infrastructure-.patch application/octet-stream 95.2 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message David Geier 2026-10-26 15:25:22 Re: Hash index inserts serialise on the metapage tuple counter
Previous Message Andres Freund 2026-10-26 15:15:04 Re: WAL record CRC calculated incorrectly because of underlying buffer modification