Re: Proposal: Conflict log history table for Logical Replication

From: Nisha Moond <nisha(dot)moond412(at)gmail(dot)com>
To: shveta malik <shveta(dot)malik(at)gmail(dot)com>
Cc: Zhijie Hou <houzhijie22(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Dilip Kumar <DilipBalaut(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>, saurabh singh <saurabh(dot)singh214(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Bharath Rupireddy <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Proposal: Conflict log history table for Logical Replication
Date: 2026-10-16 14:35:21
Message-ID: CABdArM53YBm8uenAsFunOf6MZqDP+N8JBNbmYBdPyoM1Q0eyLg@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Oct 14, 2026 at 9:56 AM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> On Wed, Oct 14, 2026 at 7:43 AM Zhijie Hou <houzhijie22(at)gmail(dot)com> wrote:
> >
> > Hi,
> >
> > On Tue, Oct 13, 2026 at 6:29 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
> > >
> > > I analyzed that 'intervalstyle' could also cause correctness issue
> > > when reading the values. Attached the testcase with results before and
> > > after fix in it.
> > >
> > > The fix would be similar to DateStyle fix in build_index_key_json():
> > >
> > > + (void) set_config_option("intervalstyle", "postgres",
> > > + PGC_USERSET, PGC_S_SESSION,
> > > + GUC_ACTION_SAVE, true, 0, false);
> > >
> > > Will analyze 'extra_float_digits' next.
> >
> > Thanks!
> >
> > I also re-confirmed locally that all the parameters you reported (e.g.
> > DateStyle, IntervalStyle, extra_float_digits) have similar issues, and setting
> > the GUCs manually when building the CLT row is an appropriate way to fix this.
>
> Thanks for verifying.
>
> > Please see the attachment for a TAP test on top of v81 that tries to reproduce
> > all of them. And if we plan to fix this along with the main patch, I think it
> > would be worth keeping a simple test case like the attached one.
> >
>
> I like the idea of test-patch. But the test has a few problems. It
> did not compile and run on my machine. Problems were:
>
> 1)
> [09:09:53] t/035_conflicts.pl .. Global symbol "$node_A" requires
> explicit package name (did you forget to declare "my $node_A"?) at
> t/035_conflicts.pl line 209.
> Execution of t/035_conflicts.pl aborted due to compilation errors.
>
> Problem was this part:
> +# Wait for the apply worker to stop
> +$node_A->poll_query_until('postgres',
> + "SELECT count(*) = 0 FROM pg_stat_activity WHERE backend_type =
> 'logical replication apply worker'"
>
> I think we need to change node_A to node_subscriber. Otherwise also,
> it was wrong as node_A is assigned as:
>
> my $node_A = $node_publisher
>
> And we want to stop worker on subscriber.
>
>
> 2)
> After fixing above, the testcase fails for me:
>
> [09:12:35] t/035_conflicts.pl .. 6/?
> # Failed test 'CLT JSON stores timestamp in ISO format'
> # at t/035_conflicts.pl line 239.
> # '2026-03-04 10:11:12.123456+05:30'
> # doesn't match '(?^:^\d{4}-\d{2}-\d{2}
> \d{2}:\d{2}:\d{2}(\.\d+)?[+-]\d{2}$)'
>
> This is because it could not match timezone offsets that include
> minutes (e.g. +05:30 in IST) on my machine. This needed to be changed
> to:
> qr/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}(\.\d+)?[+-]\d{2}(:\d{2}){0,2}$/,
>
> ~~
>
> I have addressed these 2 test-issues in the attached patch.
>
> Also added code to fix intervalstyle and extra_float issues. Please review once.
>
> Attached patch applies atop v81-002.
>

Thanks, Shveta. I merged this into v82-0003.
The test hung on my machine. Once the new test cleared the conf_tab_2
error, sub_tab's apply worker stayed running, so a later wait for all
apply workers to stop never finished. I added a step to disable
sub_tab at the end of the new test to fix it.

--
Thanks,
Nisha

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2026-10-16 14:38:49 Re: [PG19] pg_upgrade from <= 18 loses the newest multixid if its members end at offset wraparound
Previous Message Nisha Moond 2026-10-16 14:34:19 Re: Proposal: Conflict log history table for Logical Replication