Re: Proposal: Conflict log history table for Logical Replication

From: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
To: Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Cc: shveta malik <shveta(dot)malik(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2025-12-19 08:27:24
Message-ID: CAD21AoD5CumuHoHVduAboYTE95i7-n3G0b3cy6Ux+XHHdpKtfA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Dec 18, 2025 at 10:24 PM Dilip Kumar <dilipbalaut(at)gmail(dot)com> wrote:
>
> On Fri, Dec 19, 2025 at 10:40 AM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> > > 1. What should be the name of the option 'conflict_log_destination' vs
> > > 'conflict_log_format'
> >
> > I prefer conflcit_log_destination.
> >
> > > 2. Do we want to support multi destination then providing string like
> > > 'conflict_log_destination = 'log,table,..' make more sense but then we
> > > would have to store as a string in catalog and parse it everytime we
> > > insert conflicts or alter subscription OTOH currently I have just
> > > support single option log/table/both which make things much easy
> > > because then in catalog we can store as a single char field and don't
> > > need any parsing. And since the input are taken as a string itself,
> > > even if in future we want to support more options like 'log,table,..'
> > > it would be backward compatible with old options.
> >
> > I feel, combination of options might be a good idea, similar to how
> > 'log_destination' provides. But it can be done in future versions and
> > the first draft can be a simple one.
> >
> > > 3. Do we want to support 'none' destinations? i.e. do not log to anywhere?
> >
> > IMO, conflict information is an important piece of information to
> > diagnose data divergence and thus should be logged always.
> >
> > Let's wait for others' opinions.
>
> Thanks Shveta for you opinion,
>
> Here is what I propose considering balance between simplicity with
> future scalability:
>
> 1. Retain 'conflict_log_destination' as the option name.
> 2. Current supported values include 'log', 'table', or 'all' (which
> directs output to both locations). But we will not support comma
> separated values in the first version.

If users set conflict_log_destination='table', we don't report
anything related to conflict to the server logs while all other errors
generated by apply workers go to the server logs? or do we write
ERRORs without the conflict details while writing full conflict logs
to the table? If we go with the former idea, monitoring tools would
not be able to catch ERROR logs. Users can set
conflict_log_destination='all' in this case, but they might want to
avoid bloating the server logs by the detailed conflict information. I
wonder if there might be cases where monitoring tools want to detect
at least the fact that errors occur in the system.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2025-12-19 08:47:24 Re: POC: enable logical decoding when wal_level = 'replica' without a server restart
Previous Message Bertrand Drouvot 2025-12-19 08:23:24 Re: Fix memory leak in gist_page_items() of pageinspect