RE: Perform streaming logical transactions by background workers and parallel apply

On Friday, October 14, 2022 12:30 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Oct 12, 2022 at 7:41 AM wangw(dot)fnst(at)fujitsu(dot)com
> <wangw(dot)fnst(at)fujitsu(dot)com> wrote:
> >
> > On Fri, Oct 7, 2022 at 14:18 PM Hou, Zhijie/侯 志杰
> <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com> wrote:
> > > Attach the new version patch set which addressed most of the comments.
> >
> > Rebased the patch set because the new change in HEAD (776e1c8).
> >
> > Attach the new patch set.
> >
>
> +static void
> +HandleParallelApplyMessage(ParallelApplyWorkerInfo *winfo, StringInfo
> +msg)
> {
> ...
> + case 'X': /* Terminate, indicating clean exit */ {
> + shm_mq_detach(winfo->error_mq_handle);
> + winfo->error_mq_handle = NULL;
> + break;
> + }
> ...
> }
>
> I don't see the use of this message in the patch. If this is not required by the
> latest version then we can remove it and its corresponding handling in
> parallel_apply_start_worker(). I am referring to the below code in
> parallel_apply_start_worker():

Thanks for the comments, I removed these codes in the new version patch set.

I also did the following changes in the new version patch:

[0001]
* Teach the parallel apply worker to catch the subscription parameter change in
the main loop so that user can change the streaming option to "on" to stop
the parallel apply workers in case the leader apply workers get stuck because of
some deadlock problems discussed in [1].

* Some cosmetic changes.

* Address comments from Peter[2].

[0004]
* Disallow replicating from or to a partitioned table in parallel streaming
mode. This is to avoid the deadlock cases when the partitioned table's
inheritance structure is different between publisher and subscriber as
discussed [1].

[1] https://www.postgresql.org/message-id/CAA4eK1JYFXEoFhJAvg1qU%3DnZrZLw_87X%3D2YWQGFBbcBGirAUwA%40mail.gmail.com
[2] https://www.postgresql.org/message-id/CAHut%2BPvxL8tJ2ZUpEjkbRFe6qKSH%2Br54BQ7wM8p%3D335tUbuXbg%40mail.gmail.com

Best regards,
Hou zj