Timing issue in pg_basebackup

From: Jesper St John <jesper(at)stjohn(dot)se>
To: pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Timing issue in pg_basebackup
Date: 2026-10-19 14:33:14
Message-ID: ddfa03dcf26f366ae493e8cc41bd479a@stjohn.se
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hey!

I recently came across this issue when running pg_basebackup from 2
nodes at the same time through ansible. One of the nodes sometimes
reported: "requested WAL segment ... has already been removed". Tested
on v16, v17 and v18. The workaround was running them in sequence or
pause between runs.

It looks like there's a timing issue here:
1. Backup A starts and runs a checkpoint - needs WAL from segment 2.
2. Backup A hasn't reserved that WAL yet.
3. Backup B starts and runs a checkpoint (or anyone runs a checkpoint) -
deletes segment 2.
4. Backup A asks for segment 2, which is gone, and fails.

I've included a patch that moves the replication slot handing to run
before the backup part as a separate function.

Some possible caveats already:
* Since moving the slot creation, we might run in to trouble if for
example, --checkpoint=spread is used since the slot is open for longer.
* If any error arises in the backup part, now there will be a lingering
slot when previously no slot was created.

*** DISCLAIMER ***
I've used an LLM to help me write the test - I'm rubbish at Perl.

Love to get some feedback.

Regards,
Jesper St John

Attachment Content-Type Size
v1-0001-pg_basebackup-Create-the-WAL-streaming-slot-befor.patch text/x-diff 9.9 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Henson Choi 2026-10-19 14:35:03 Re: BUG #19767: pg_get_viewdef prints ORDER BY (- '3'::int4) as (- 3), which reparses as column position -3
Previous Message Peter Eisentraut 2026-10-19 12:45:56 Re: BUG #19644: byteaout, float8out and float4out are marked IMMUTABLE but depend on GUCs