Re: Recovery conflict resolution misses backends that import snapshots

From: "chee(dot)wooson" <chee(dot)wooson(at)gmail(dot)com>
To: Scott Ray <scott(at)scottray(dot)io>
Cc: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Michael Paquier <michael(at)paquier(dot)xyz>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com>, Zhijie Hou <houzj(dot)fnst(at)fujitsu(dot)com>, Hayato Kuroda <kuroda(dot)hayato(at)fujitsu(dot)com>, Bertrand Drouvot <bertranddrouvot(dot)pg(at)gmail(dot)com>
Subject: Re: Recovery conflict resolution misses backends that import snapshots
Date: 2026-09-11 02:51:30
Message-ID: 20260911025213.1156239-1-chee.wooson@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Based on the discussion, I tried an alternative approach for master that
avoids repeatedly rescanning the procarray.

Recovery currently collects a fixed list of VXIDs whose xmins conflict
with a cleanup WAL record. After ProcArrayLock is released, another
backend can import a listed backend's snapshot and advertise the same
xmin. The importer is not in recovery's wait list, so recovery can
finish waiting and replay the cleanup while the imported snapshot still
needs the removed data.

The attached v3 uses a separate atomic recoveryConflictTracked field in
PGPROC. Startup marks each conflicting snapshot source while collecting
the wait list under shared ProcArrayLock. ProcArrayInstallImportedXmin()
holds ProcArrayLock exclusively and rejects imports from marked sources.

This gives the following ordering:

- An import completed before the scan is visible to the scan and included
in the wait list.
- An import attempted after the scan observes the source marker and fails.

ResolveRecoveryConflictWithVirtualXIDs() clears each marker immediately
after the corresponding VXID finishes. The marker is used only for
RECOVERY_CONFLICT_SNAPSHOT. It is separate from pendingRecoveryConflicts
because the cancellation bits have a different lifetime and are consumed
by backend interrupt processing.

Compared with v2, this prevents the chain of conflicting importers from
growing instead of rescanning until no conflicts remain. It also retains
the boolean return value of ProcArrayInstallImportedXmin(), and the new
field does not need explicit initialization in ProcGlobalShmemInit, just
as pendingRecoveryConflicts does not.

The attached series is:

- v3-0001 adds a deterministic TAP reproducer and its injection points.
- v3-0002 implements the recoveryConflictTracked protocol.

Patch 0001 is expected to fail without patch 0002 because the conflicting
snapshot import succeeds.

The series is based on master at 9f4bd91a196. I tested it with assertions,
injection points, and TAP tests enabled. The build completed successfully,
and recovery tests 056_standby_snapshot_export and
057_snapshot_import_conflict passed. The series also applies cleanly to
that master commit.

This approach conservatively rejects all snapshot imports from a tracked
source until its tracked VXID finishes. Feedback on this tradeoff and the
marker lifetime would be appreciated.

Regards,
Chee

Attachment Content-Type Size
v3-0001-Add-TAP-test-for-recovery-conflicts-from-imported.patch text/x-patch 6.7 KB
v3-0002-Fix-recovery-conflict-resolution-to-account-for-i.patch text/x-patch 8.6 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Chao Li 2026-09-11 03:02:29 Re: eliminate xl_heap_visible to reduce WAL (and eventually set VM on-access)
Previous Message Hayato Kuroda (Fujitsu) 2026-09-11 02:51:01 RE: pg_createsubscriber does not check output_plugin_libraries