RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication

From: "Hayato Kuroda (Fujitsu)" <kuroda(dot)hayato(at)fujitsu(dot)com>
To: 'Amit Kapila' <amit(dot)kapila16(at)gmail(dot)com>, Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>
Cc: Peter Smith <smithpb2250(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, "Wei Wang (Fujitsu)" <wangw(dot)fnst(at)fujitsu(dot)com>, "Yu Shi (Fujitsu)" <shiy(dot)fnst(at)fujitsu(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, shveta malik <shveta(dot)malik(at)gmail(dot)com>
Subject: RE: [PATCH] Reuse Workers and Replication Slots during Logical Replication
Date: 2023-06-28 06:31:54
Message-ID: TYAPR01MB586659C7EAE9A27553512B54F524A@TYAPR01MB5866.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dear Amit,

> > > This actually makes sense. I quickly try to do that without adding any
> > > new replication message. As you would expect, it did not work.
> > > I don't really know what's needed to make a connection to last for
> > > more than one iteration. Need to look into this. Happy to hear any
> > > suggestions and thoughts.
> >
>
> It is not clear to me what exactly you tried here which didn't work.
> Can you please explain a bit more?

Just to confirm, this is not my part. Melih can answer this...

> > I have analyzed how we handle this. Please see attached the patch (0003) which
> > allows reusing connection.
> >
>
> Why did you change the application name during the connection?

It was because the lifetime of tablesync worker is longer than slots's one and
tablesync worker creates temporary replication slots many times, per the target
relation. The name of each slots has relid, so I thought that it was not suitable.
But in the later patch the tablesync worker tries to reuse the slot during the
synchronization, so in this case the application_name should be same as slotname.

I added comment in 0003, and new file 0006 file to use slot name as application_name
again. Note again that the separation was just for specifying changes, Melih can
include them to one part of files if needed.

Best Regards,
Hayato Kuroda
FUJITSU LIMITED

Attachment Content-Type Size
0005-Reuse-Replication-Slot-and-Origin-in-Tablesync.patch application/octet-stream 55.8 KB
0006-Use-slot-name-as-application_name-again.patch application/octet-stream 1.6 KB
0001-Refactor-to-split-Apply-and-Tablesync-Workers.patch application/octet-stream 21.1 KB
0002-Reuse-Tablesync-Workers.patch application/octet-stream 10.4 KB
0003-reuse-connection-when-tablesync-workers-change-the-t.patch application/octet-stream 4.3 KB
0004-Add-replication-protocol-cmd-to-create-a-snapshot.patch application/octet-stream 21.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Drouvot, Bertrand 2023-06-28 06:49:50 Re: Synchronizing slots from primary to standby
Previous Message jian he 2023-06-28 06:26:52 Re: Do we want a hashset type?