Re: Force streaming every change in logical decoding

On Fri, Dec 23, 2022 at 2:03 PM shiy(dot)fnst(at)fujitsu(dot)com <shiy(dot)fnst(at)fujitsu(dot)com>
wrote:
>
> On Fri, Dec 23, 2022 1:50 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> >
> > On Thu, Dec 22, 2022 at 6:18 PM shiy(dot)fnst(at)fujitsu(dot)com
> > <shiy(dot)fnst(at)fujitsu(dot)com> wrote:
> > >
> > >
> > > Besides, I tried to reduce data size in streaming subscription tap
tests by this
> > > new GUC (see 0002 patch). But I didn't covert all streaming tap tests
> > because I
> > > think we also need to cover the case that there are lots of changes.
So, 015*
> > is
> > > not modified. And 017* is not modified because streaming transactions
and
> > > non-streaming transactions are tested alternately in this test.
> > >
> >
> > I think we can remove the newly added test from the patch and instead
> > combine the 0001 and 0002 patches. I think we should leave the
> > 022_twophase_cascade as it is because it can impact code coverage,
> > especially the below part of the test:
> > # 2PC PREPARE with a nested ROLLBACK TO SAVEPOINT
> > $node_A->safe_psql(
> > 'postgres', "
> > BEGIN;
> > INSERT INTO test_tab VALUES (9999, 'foobar');
> > SAVEPOINT sp_inner;
> > INSERT INTO test_tab SELECT i, md5(i::text) FROM
> > generate_series(3, 5000) s(i);
> >
> > Here, we will stream first time after the subtransaction, so can
> > impact the below part of the code in ReorderBufferStreamTXN:
> > if (txn->snapshot_now == NULL)
> > {
> > ...
> > dlist_foreach(subxact_i, &txn->subtxns)
> > {
> > ReorderBufferTXN *subtxn;
> >
> > subtxn = dlist_container(ReorderBufferTXN, node, subxact_i.cur);
> > ReorderBufferTransferSnapToParent(txn, subtxn);
> > }
> > ...
> >
>
> OK, I removed the modification in 022_twophase_cascade.pl and combine the
two patches.
>
> Please see the attached patch.
> I also fixed Kuroda-san's comments[1].
>
> [1]
https://www.postgresql.org/message-id/TYAPR01MB5866CD99CF86EAC84119BC91F5E99%40TYAPR01MB5866.jpnprd01.prod.outlook.com
>
> Regards,
> Shi yu

Hello,
I ran tests (4 runs) on both versions (v5 and v6) in release mode. And the
data looks promising, time is reduced now:

HEAD V5
Delta (sec)
2:20.535307 2:15.865241
4.670066
2:19.220917 2:14.445312
4.775605
2:22.492128 2:17.35755
5.134578
2:20.737309 2:15.564306
5.173003

HEAD V6
Delta (sec)
2:20.535307 2:15.363567
5.17174
2:19.220917 2:15.079082
4.14.1835
2:22.492128 2:16.244139
6.247989
2:20.737309 2:16.108033
4.629276

thanks
Shveta