Re: Perform streaming logical transactions by background workers and parallel apply

On Fri, Dec 23, 2022 at 12:21 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Thu, Dec 22, 2022 at 6:18 PM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> >
> > On Thu, Dec 22, 2022 at 7:04 PM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
> > >
> > > On Thu, Dec 22, 2022 at 11:39 AM Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> wrote:
> > > >
> > > > Thank you for updating the patch. Here are some comments on v64 patches:
> > > >
> > > > While testing the patch, I realized that if all streamed transactions
> > > > are handled by parallel workers, there is no chance for the leader to
> > > > call maybe_reread_subscription() except for when waiting for the next
> > > > message. Due to this, the leader didn't stop for a while even if the
> > > > subscription gets disabled. It's an extreme case since my test was
> > > > that pgbench runs 30 concurrent transactions and logical_decoding_mode
> > > > = 'immediate', but we might want to make sure to call
> > > > maybe_reread_subscription() at least after committing/preparing a
> > > > transaction.
> > > >
> > >
> > > Won't it be better to call it only if we handle the transaction by the
> > > parallel worker?
> >
> > Agreed. And we won't need to do that after handling stream_prepare as
> > we don't do that now.
> >
>
> I think we do this for both prepare and non-prepare cases via
> begin_replication_step(). Here, in both cases, as the changes are sent
> to the parallel apply worker, we missed in both cases. So, I think it
> is better to do in both cases.

Agreed. I missed that we call maybe_reread_subscription() even in the
prepare case.

Regards,

--
Masahiko Sawada
Amazon Web Services: https://aws.amazon.com