RE: Perform streaming logical transactions by background workers and parallel apply

From: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, shveta malik <shveta(dot)malik(at)gmail(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>
Subject: RE: Perform streaming logical transactions by background workers and parallel apply
Date: 2023-01-13 10:13:25
Message-ID: OS0PR01MB5716199216FFD4AED083BA8294C29@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Friday, January 13, 2023 1:02 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
> On Fri, Jan 13, 2023 at 1:28 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> >
> > On Fri, Jan 13, 2023 at 9:06 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
> wrote:
> > >
> > > On Fri, Jan 13, 2023 at 7:56 AM Peter Smith <smithpb2250(at)gmail(dot)com>
> wrote:
> > > >
> > >
> > > >
> > > > 3.
> > > >
> > > > <entry role="catalog_table_entry"><para
> > > > role="column_definition">
> > > > + <structfield>leader_pid</structfield> <type>integer</type>
> > > > + </para>
> > > > + <para>
> > > > + Process ID of the leader apply worker if this process is a parallel
> > > > + apply worker; NULL if this process is a leader apply worker or
> does not
> > > > + participate in parallel apply, or a synchronization worker
> > > > + </para></entry>
> > > >
> > > > I felt this change is giving too many details and ended up just
> > > > muddying the water.
> > > >
> > >
> > > I see that we give a similar description for other parameters as well.
> > > For example leader_pid in pg_stat_activity,
> > >
> >
> > BTW, shouldn't we update leader_pid column in pg_stat_activity as well
> > to display apply leader PID for parallel apply workers? It will
> > currently display for other parallel operations like a parallel
> > vacuum, so I don't see a reason to not do the same for parallel apply
> > workers.
>
> +1
>
> The parallel apply workers have different properties than the parallel query
> workers since they execute different transactions and don't use group locking
> but it would be a good hint for users to show the leader and parallel apply
> worker processes are related. If users want to check only parallel query workers
> they can use the backend_type column.

Agreed, and changed as suggested.

Attach the new version patch set which address the comments so far.

Best Regards,
Hou zj

Attachment Content-Type Size
v80-0004-Retry-to-apply-streaming-xact-only-in-apply-work.patch application/octet-stream 21.1 KB
v80-0001-Display-the-leader-apply-worker-s-PID-for-parall.patch application/octet-stream 15.4 KB
v80-0002-Stop-extra-worker-if-GUC-was-changed.patch application/octet-stream 4.2 KB
v80-0003-Add-GUC-stream_serialize_threshold-and-test-seri.patch application/octet-stream 13.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message houzj.fnst@fujitsu.com 2023-01-13 10:13:31 RE: Perform streaming logical transactions by background workers and parallel apply
Previous Message Dean Rasheed 2023-01-13 10:01:15 Re: Non-decimal integer literals