Re: Proposal: Conflict log history table for Logical Replication

From: shveta malik <shveta(dot)malik(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-08-11 05:18:40
Message-ID: CAJpy0uAZtZQhLsZiPrv59LkUQ4a_708ofRQimJLh5+ya2Ru-gg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Mon, Aug 10, 2026 at 6:45 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Mon, Aug 10, 2026 at 10:32 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Sat, Aug 8, 2026 at 2:23 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > >
> > > There are two possible definitions we could give to CLT:
> > >
> > > 1. it stores how replication conflicts were resolved.
> > > 2. it stores which replication conflicts caused data divergences (and
> > > how they were resolved).
> > >
> > > My interpretation had been (1). The option is named
> > > conflict_log_destination, which reads as choosing where the same
> > > information goes, and the server log carries every conflict including
> > > the conflicts resolved by the ERROR resolver. So I expected the table
> > > to have the same coverage.
> > >
> > > That said, I can accept (2), and I agree it is what other replication
> > > solutions do. If we go that way, though, I would like the CLT to have
> > > a 'resolution' column, showing which resolution method was applied,
> > > even though the resolvers are hard-wired per conflict type today. With
> > > that column, a user can see both what conflict happened and how it was
> > > handled by looking at the CLT alone.
>
> Here is an updated patch as we discussed, and below is the list of changes:
>
> 1. Insert only resolved (LOG-level) conflicts in the conflict log
> table: Conflicts that raise an ERROR halt replication and roll back
> the transaction, so they are logged exclusively to the server log for
> monitoring and alerting. The table is preserved purely as an audit log
> for resolved conflicts where data is updated or skipped, I updated the
> documentation accordingly.
>
> 2. Removed the key column from LocalConflictSchema: Since unique index
> violations are ERROR-level and no longer stored in the table, the key
> attribute is unnecessary and has been removed.
>
> 3. Preserved local_conflicts as json[]: Although currently recorded
> conflicts involve a single local tuple, we decided to keep
> local_conflicts as a JSON array for future-proofing. This ensures that
> any future enhancements (such as multi-row conflict resolvers) will
> not require backward-incompatible table schema changes.
>
> I will work on other follow-up patches soon.
>

+1 on the idea of skipping logging ERROR-level conflicts to the CLT
for the reasons discussed above.

Initial feedback on the patch:

--I verified the patch. All non-ERROR conflict types are correctly
logged to both the log file and the CLT through the apply worker.
ERROR-level conflicts are logged only to the log file, as discussed
above.

--The parallel apply worker no longer has the old problem caused by
ERROR-level conflicts. I also verified a few other conflict types
(though not all) through parallel apply, and they are logged correctly
to the CLT.

I’ll verify further and review the changed in more detail soon.

thanks
Shveta

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message John Naylor 2026-08-11 05:35:30 Re: arrays over initdb-created types are broken after pg_upgrade
Previous Message Masashi Kamura (Fujitsu) 2026-08-11 05:16:21 Crash issue in PG18.5 regression