Re: Perform streaming logical transactions by background workers and parallel apply

From: Peter Smith <smithpb2250(at)gmail(dot)com>
To: Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>
Cc: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, "wangw(dot)fnst(at)fujitsu(dot)com" <wangw(dot)fnst(at)fujitsu(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, "shiy(dot)fnst(at)fujitsu(dot)com" <shiy(dot)fnst(at)fujitsu(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Perform streaming logical transactions by background workers and parallel apply
Date: 2022-11-18 02:30:43
Message-ID: CAHut+Pvvv73jt6eKtbtgzC6ZMWxAWvCAMFP=wj5t+V39=C_rEw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Fri, Nov 18, 2022 at 11:36 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
>
...
> ---
> The streaming parameter has the new value "parallel" for "streaming"
> option to enable the parallel apply. It fits so far but I think the
> parallel apply feature doesn't necessarily need to be tied up with
> streaming replication. For example, we might want to support parallel
> apply also for non-streaming transactions in the future. It might be
> better to have another option, say "parallel", to control parallel
> apply behavior. The "parallel" option can be a boolean option and
> setting parallel = on requires streaming = on.
>

FWIW, I tend to agree with this idea but for a different reason. In
this patch, the 'streaming' parameter had become a kind of hybrid
boolean/enum. AFAIK there are no other parameters anywhere that use a
hybrid pattern like this so I was thinking it may be better not to be
different.

But I didn't think that parallel_apply=on should *require*
streaming=on. It might be better for parallel_apply=on is just the
*default*, but it simply achieves nothing unless streaming=on too.
That way users would not need to change anything at all to get the
benefits of parallel streaming.

> Another variant is to have a new subscription parameter for example
> "parallel_workers" parameter that specifies the number of parallel
> workers. That way, users can specify the number of parallel workers
> per subscription.
>

------
Kind Regards,
Peter Smith.
Fujitsu Australia.

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2022-11-18 03:03:25 Re: logical decoding and replication of sequences, take 2
Previous Message Michael Paquier 2022-11-18 02:27:41 Re: Fix some newly modified tab-complete changes