Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

On Mon, Jul 3, 2023 at 9:42 AM Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> wrote:
>
> On Wed, Jun 28, 2023 at 12:02 PM Hayato Kuroda (Fujitsu)
> <kuroda(dot)hayato(at)fujitsu(dot)com> wrote:
>
> > But in the later patch the tablesync worker tries to reuse the slot during the
> > synchronization, so in this case the application_name should be same as slotname.
> >
>
> Fair enough. I am slightly afraid that if we can't show the benefits
> with later patches then we may need to drop them but at this stage I
> feel we need to investigate why those are not helping?
>

On thinking about this, I think the primary benefit we were expecting
by saving network round trips for slot drop/create but now that we
anyway need an extra round trip to establish a snapshot, so such a
benefit was not visible. This is just a theory so we should validate
it. The another idea as discussed before [1] could be to try copying
multiple tables in a single transaction. Now, keeping a transaction
open for a longer time could have side-effects on the publisher node.
So, we probably need to ensure that we don't perform multiple large
syncs and even for smaller tables (and later sequences) perform it
only for some threshold number of tables which we can figure out by
some tests. Also, the other safety-check could be that anytime we need
to perform streaming (sync with apply worker), we won't copy more
tables in same transaction.

Thoughts?

[1] - https://www.postgresql.org/message-id/CAGPVpCRWEVhXa7ovrhuSQofx4to7o22oU9iKtrOgAOtz_%3DY6vg%40mail.gmail.com

--
With Regards,
Amit Kapila.