RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>
Cc: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, Peter Smith <smithpb2250(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, "Wei Wang (Fujitsu)" <wangw(dot)fnst(at)fujitsu(dot)com>, "Yu Shi (Fujitsu)" <shiy(dot)fnst(at)fujitsu(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Date: 2023-07-04 05:42:48
Message-ID: TYAPR01MB58663E4944DCBCCF4AD14EE0F52EA@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Amit,

> > > > I have analyzed how we handle this. Please see attached the patch (0003)
> which
> > > > allows reusing connection.
> > > >
> > >
> > > Why did you change the application name during the connection?
> >
> > It was because the lifetime of tablesync worker is longer than slots's one and
> > tablesync worker creates temporary replication slots many times, per the target
> > relation. The name of each slots has relid, so I thought that it was not suitable.
> >
>
> Okay, but let's try to give a unique application name to each
> tablesync worker for the purpose of pg_stat_activity and synchronous
> replication (as mentioned in existing comments as well). One idea is
> to generate a name like pg_<sub_id>_sync_<worker_slot> but feel free
> to suggest if you have any better ideas.

Good point. The slot id is passed as an argument of TablesyncWorkerMain(),
so I passed it to LogicalRepSyncTableStart(). PSA new set.

> > But in the later patch the tablesync worker tries to reuse the slot during the
> > synchronization, so in this case the application_name should be same as
> slotname.
> >
>
> Fair enough. I am slightly afraid that if we can't show the benefits
> with later patches then we may need to drop them but at this stage I
> feel we need to investigate why those are not helping?

Agreed. Now I'm planning to do performance testing independently. We can discuss
based on that or Melih's one.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
v2-0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch application/octet-stream 21.1 KB
v2-0002-Reuse-Tablesync-Workers.patch application/octet-stream 10.4 KB
v2-0003-reuse-connection-when-tablesync-workers-change-th.patch application/octet-stream 6.9 KB
v2-0004-Add-replication-protocol-cmd-to-create-a-snapshot.patch application/octet-stream 21.0 KB
v2-0005-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch application/octet-stream 56.0 KB
v2-0006-Use-slot-name-as-application_name-again.patch application/octet-stream 1.6 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2023-07-04 05:48:39 Re: [PoC] Improve dead tuple storage for lazy vacuum
Previous Message Noah Misch 2023-07-04 05:19:43 Re: pgsql: Fix search_path to a safe value during maintenance operations.