| From: | "Zhijie Hou (Fujitsu)" <houzj(dot)fnst(at)fujitsu(dot)com> |
|---|---|
| To: | Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> |
| Cc: | suyu(dot)cmj <mengjuan(dot)cmj(at)alibaba-inc(dot)com>, tomas <tomas(at)vondra(dot)me>, michael <michael(at)paquier(dot)xyz>, bharath(dot)rupireddyforpostgres <bharath(dot)rupireddyforpostgres(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Alexander Korotkov <aekorotkov(at)gmail(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Vitaly Davydov <v(dot)davydov(at)postgrespro(dot)ru> |
| Subject: | RE: Newly created replication slot may be invalidated by checkpoint |
| Date: | 2025-11-19 11:40:31 |
| Message-ID: | TY4PR01MB16907FF0F8D547A9C56B3975C94D7A@TY4PR01MB16907.jpnprd01.prod.outlook.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Wednesday, November 19, 2025 4:24 PM Hou, Zhijie<houzj(dot)fnst(at)fujitsu(dot)com> wrote:
>
> On Monday, November 17, 2025 6:50 PM Zhijie Hou (Fujitsu)
> <houzj(dot)fnst(at)fujitsu(dot)com> wrote:
> >
> >
> > Attach v2 patch that improves the fix based on above analysis.
> >
> > (I am still testing some other scenarios related to slotsync to ensure the
> > current fix is complete)
>
> I have been testing whether taking exclusive ReplicationSlotAllocationLock
> can
> prevent newly synced slot from being invalidated, but found that it is
> insufficient.
>
> During slotsync, since it fetches the remote LSN (queried from the publisher)
> as
> the initial restart_lsn for newly synced slot and we sync the slots in a
> asynchronous manner, the synced restart_lsn could be behind the standby's
> redo
> pointer. So, there could be race conditions that the checkpoint removes the
> required WALs and invalidates the newly synced slot:
>
> 1. Assuming there is no slot on standby, the minimum slot LSN would be
> invalid,
> and the checkpoint reaches InvalidateObsoleteReplicationSlots() with
> oldestSegno =segno of redo.
> 2. The slotsync successfully sync the slot A that has old restart_lsn.
> 3. The checkpoint would then find that the newly synced slot A should be
> invalidated due to old restart_lsn.
..
>
> Here is the V3 patch that fixes all the race conditions found so far.
I am attaching a tap-test in 0002 that includes an injection point to reproduce
this scenario. The test expects that while syncing a slot to the standby server,
if the WAL before the remote restart_lsn is at risk of being removed by a
checkpoint, the slot cannot be synced. Without the fix in 0001, the test would
fail because the slot is synced to standby but immediately invalidated by the
checkpoint.
I am not adamant about merging this test but just for reference.
Besides, I fixed a bug in v3-0001 where the restart_lsn is set to a wrong value.
>
> ------
>
> Apart from addressing the fix for HEAD, I would like to inquire if anyone holds
> a differing opinion regarding the revert of the origin fix 2090edc6f32f652a2c in
> the back branches and applying the same fix as HEAD.
>
> I searched for extensions that rely on the size of ReplicationSlot and did not
> find any, so I think it is OK to implement the same fix in the backpatches.
>
> It there are no alternative views, I will proceed with preparing the patches for
> the back branches in upcoming versions.
Best Regards,
Hou zj
| Attachment | Content-Type | Size |
|---|---|---|
| v4-0002-Add-a-tap-test-using-injection-point.patch | application/octet-stream | 4.3 KB |
| v4-0001-Fix-race-conditions-causing-invalidation-of-newly.patch | application/octet-stream | 15.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Fujii Masao | 2025-11-19 11:46:25 | Re: Exit walsender before confirming remote flush in logical replication |
| Previous Message | Peter Eisentraut | 2025-11-19 11:23:25 | Re: Trying out <stdatomic.h> |