Re: [PATCH] Reuse Workers and Replication Slots during Logical Replication

On Fri, Jul 14, 2023 at 1:58 AM Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com> wrote:
>
> Here are some quick numbers with 100 empty tables.
>
> +--------------+----------------+----------------+----------------+
> | | 2 sync workers | 4 sync workers | 8 sync workers |
> +--------------+----------------+----------------+----------------+
> | POC design#1 | 1909.873 ms | 986.261 ms | 552.404 ms |
> +--------------+----------------+----------------+----------------+
> | POC design#2 | 4962.208 ms | 1240.503 ms | 1165.405 ms |
> +--------------+----------------+----------------+----------------+
> | master | 2666.008 ms | 1462.012 ms | 986.848 ms |
> +--------------+----------------+----------------+----------------+
>
> Seems like design#1 is better than both design#2 and master overall. It's surprising to see that even master beats design#2 in some cases though. Not sure if that is expected or there are some places to improve design#2 even more.
>

Yeah, it is quite surprising that Design#2 is worse than master. I
suspect there is something wrong going on with your Design#2 patch.
One area to check is whether apply worker is able to quickly assign
the new relations to tablesync workers. Note that currently after the
first time assigning the tables to workers, the apply worker may wait
before processing the next set of tables in the main loop of
LogicalRepApplyLoop(). The other minor point about design#2
implementation is that you may want to first assign the allocated
tablesync workers before trying to launch a new worker.

>
> PS: I only attached the related patches and not the whole patch set. 0001 and 0002 may contain some of your earlier reviews, but I'll send a proper updated set soon.
>

Yeah, that would be helpful.

--
With Regards,
Amit Kapila.