Re: Perform streaming logical transactions by background workers and parallel apply

From: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-12-01 08:46:10
Message-ID: CAA4eK1Ja9P+KUKibF_14Tv-+-mqU9+QW2RdH6PagjrQfAX3Tvw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Dec 1, 2022 at 11:44 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Wed, Nov 30, 2022 at 7:54 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Tue, Nov 29, 2022 at 10:18 AM houzj(dot)fnst(at)fujitsu(dot)com
> > <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> > >
> > > Attach the new version patch which addressed all comments.
> > >
> >
> > Some comments on v53-0002*
> > ========================
> > 1. I think testing the scenario where the shm_mq buffer is full
> > between the leader and parallel apply worker would require a large
> > amount of data and then also there is no guarantee. How about having a
> > developer GUC [1] force_apply_serialize which allows us to serialize
> > the changes and only after commit the parallel apply worker would be
> > allowed to apply it?
>
> +1
>
> The code coverage report shows that we don't cover the partial
> serialization codes. This GUC would improve the code coverage.
>

Shall we keep it as a boolean or an integer? Keeping it as an integer
as suggested by Kuroda-San [1] would have an added advantage that we
can easily test the cases where serialization would be triggered after
sending some changes.

[1] - https://www.postgresql.org/message-id/TYAPR01MB5866160DE81FA2D88B8F22DEF5159%40TYAPR01MB5866.jpnprd01.prod.outlook.com

--
With Regards,
Amit Kapila.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message John Naylor 2022-12-01 08:49:09 Re: [PoC] Improve dead tuple storage for lazy vacuum
Previous Message Peter Eisentraut 2022-12-01 08:25:44 File API cleanup