RE: Perform streaming logical transactions by background workers and parallel apply

From: "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
Date: 2023-01-18 04:48:01
Message-ID: OS3PR01MB6275E91D744E9A3BBAD6ABBF9EC79@OS3PR01MB6275.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jan 18, 2023 12:36 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> On Tue, Jan 17, 2023 at 8:07 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
> wrote:
> >
> > On Tue, Jan 17, 2023 at 6:14 PM houzj(dot)fnst(at)fujitsu(dot)com
> > <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> > >
> > > On Tuesday, January 17, 2023 2:46 PM Masahiko Sawada
> <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > >
> > > > On Tue, Jan 17, 2023 at 12:37 PM houzj(dot)fnst(at)fujitsu(dot)com
> > > > <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> > > > I'm slightly concerned that there could be overhead of executing
> > > > GetLeaderApplyWorkerPid () for every backend process except for parallel
> > > > query workers. The number of such backends could be large and
> > > > GetLeaderApplyWorkerPid() acquires the lwlock. For example, does it
> make
> > > > sense to check (st_backendType == B_BG_WORKER) before calling
> > > > GetLeaderApplyWorkerPid()? Or it might not be a problem since it's
> > > > LogicalRepWorkerLock which is not likely to be contended.
> > >
> > > Thanks for the comment and I think your suggestion makes sense.
> > > I have added the check before getting the leader pid. Here is the new
> version patch.
> >
> > Thank you for updating the patch. Looks good to me.
> >
>
> Pushed.

Rebased and attach remaining patches for reviewing.

Regards,
Wang Wei

Attachment Content-Type Size
v84-0001-Stop-extra-worker-if-GUC-was-changed.patch application/octet-stream 4.2 KB
v84-0002-Add-GUC-stream_serialize_threshold-and-test-seri.patch application/octet-stream 13.5 KB
v84-0003-Retry-to-apply-streaming-xact-only-in-apply-work.patch application/octet-stream 21.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2023-01-18 04:59:33 Re: Remove source code display from \df+?
Previous Message Nathan Bossart 2023-01-18 04:44:27 Re: recovery modules