Re: Possible race condition in pg_basebackup

From: Nick Ivanov <nick(dot)ivanov(at)enterprisedb(dot)com>
To: Andrey Borodin <x4mmm(at)yandex-team(dot)ru>
Cc: Álvaro Herrera <alvherre(at)kurilemu(dot)de>, pgsql-hackers mailing list <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Possible race condition in pg_basebackup
Date: 2026-08-31 12:32:25
Message-ID: 999caa0e-d015-42da-9b79-001087e719b8@enterprisedb.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello Andrey,

Thank you for your comments, they are very helpful.

On 29/08/2026 15:18, Andrey Borodin wrote:
> RegisterBackupStartpoint() is called after the starting checkpoint has
> selected the startpoint. Another checkpoint can remove WAL between reading
> ControlFile and registering that LSN. The test stops later, after
> registration, so it does not exercise this window.
> ...
> Avoiding this requires registering a conservative current insert or replay
> position before requesting the starting checkpoint. If the selected
> startpoint is older, as can happen on a standby, the horizon then has to be
> lowered. Alternatively, selection and registration need an interlock with
> WAL removal.
I did consider this gap, but thought it wasn't wide enough to be of
great risk. However, you're right in that, when fixing a race condition
we should close the gap completely and not resort to a half-measure. I
will update the test and see where it takes me.
> For a backpatch, I think a new postmaster GUC is a non-starter. It adds a
> user-visible limit and shared-memory sizing decisions to a minor-version bug
> fix, while concurrent BASE_BACKUP sessions are already bounded by
> max_wal_senders. Unconditionally registering in do_pg_backup_start() also
> changes SQL-level pg_backup_start().
I didn't like introducing a new GUC myself, but I thought recycling
max_wal_senders (or max_replication_slots or whatever) would be
confusing. I believe this race condition can affect `pg_basebackup
--wal-method=fetch` as well as `--wal-method=stream`, so tying this to
replication seemed wrong to me. On the other hand, I didn't fully
consider back-porting the patch, so this needs more thought obviously.
> I suggested server-owned retention upthread, but after reading the patch I
> think your original client-side proposal deserves another look. Creating
> the requested slot before sending BASE_BACKUP directly fixes the reported
> --create-slot case, is much smaller to backpatch, and also works when a new
> pg_basebackup connects to an older server. Existing slots with a NULL
> restart_lsn, idle_replication_slot_timeout, and server-fetched WAL can be
> treated as separate server-side problems. WDYT?
I suspect this won't address the `--wal-method=fetch` situation, which I
think is subject to the same WAL removal/recycling risk.
> The September Commitfest is open for registration until September 1. I
> suggest registering the patch now so that further versions and discussion do
> not fall between CommitFests.

Thanks for the advice, will do.

Cheers

Nick

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Matthias van de Meent 2026-08-31 12:39:30 Re: Bug: Whole-row var in indexes corrupts indexes after DDL
Previous Message Andrew Dunstan 2026-08-31 12:27:15 Re: Allow table AMs to define their own reloptions