| From: | Zhijie Hou <houzhijie22(at)gmail(dot)com> |
|---|---|
| To: | "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | Masahiko Sawada <sawada(dot)mshk(at)gmail(dot)com> |
| Subject: | Fix LogStandbySnapshot() race during logical decoding activation |
| Date: | 2026-11-22 17:46:12 |
| Message-ID: | bd622b46-c734-4557-8ae1-6c113cd434a2@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
While testing the dynamic logical decoding status control, I noticed
another race condition in LogStandbySnapshot(), when logical decoding is
concurrently enabled, ProcArrayLock can be released before logging the
xl_running_xacts WAL record, causing historic snapshot corruption. The
process is as follows:
1. S1 calls GetRunningTransactionData() while T1 (XID 800, e.g., CREATE
TABLE tab_snap) is still in ProcArray, so S1 computes oldestRunningXid =
800 in memory.
2. S1 releases ProcArrayLock early, before writing its xl_running_xacts
record to WAL.
3. T1 commits: it writes COMMIT 800 at LSN 100 and finishes
ProcArrayEndTransaction(), removing 800 from ProcArray. (Had S1 kept
ProcArrayLock, T1 would have blocked here.)
4. S2 runs LogStandbySnapshot(): since 800 is already gone from
ProcArray, S2 computes oldestRunningXid = 801 (nextXid) and writes its
xl_running_xacts record at LSN 200.
5. S1 writes its delayed xl_running_xacts record at LSN 300 with the
stale oldestRunningXid = 800.
Replaying that out-of-order WAL stream corrupts SnapBuild's state:
- LSN 100 (COMMIT 800): SnapBuild adds 800 to builder->committed.xip and
sets builder->xmax = 801.
- LSN 200 (S2's RUNNING_XACTS, oldestRunningXid = 801): SnapBuild sets
builder->xmin = 801 and purges 800 from builder->committed.xip.
- LSN 300 (S1's RUNNING_XACTS, oldestRunningXid = 800): SnapBuild
overwrites builder->xmin = 800. Now 800 is back inside [xmin, xmax)
after already being purged from committed.xip, so subsequent historic
snapshots built by SnapBuild mark committed transaction 800 as still in
progress (e.g., failing with "ERROR: pg_attribute catalog is missing 1
attribute(s) for relation OID ..." or "ERROR: could not map filenumber
... to relation OID").
Currently, LogStandbySnapshot() checks IsLogicalDecodingEnabled()
(logical_decoding_enabled) to decide whether to keep ProcArrayLock.
However, enabling logical decoding happens in two phases:
1. EnableLogicalDecoding() sets xlog_logical_info = true and waits for
all processes to absorb a ProcSignalBarrier.
2. Only after that barrier wait finishes does it set
logical_decoding_enabled = true, after which the caller reserves the
new slot's restart_lsn.
If S1 absorbs the barrier in phase 1 and then calls LogStandbySnapshot()
before phase 2 sets logical_decoding_enabled = true,
IsLogicalDecodingEnabled() still returns false, so S1 releases
ProcArrayLock early. Meanwhile, since S1 has already absorbed the
barrier, the activating backend does not wait for S1: it can finish
phase 2 and reserve the new slot's restart_lsn before S1 writes its
xl_running_xacts record, exposing that out-of-order record to logical
decoding.
Attached is a patch set to fix this:
0001 checks IsXLogLogicalInfoEnabled() right before
GetRunningTransactionData() (not while holding ProcArrayLock, to respect
the lock ordering with DisableLogicalDecoding()). If xlog_logical_info
is still false at that point, neither GetRunningTransactionData() nor
LogCurrentRunningXacts() processes ProcSignalBarriers, so no concurrent
activation can finish its barrier wait until after the WAL record is
inserted.
0002 adds injection point to reproduce/test the above case. I kept this
in a separate patch because it looks complicate and might not be worth
pushing.
--
Best Regards,
Zhijie Hou
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0002-Add-test-for-LogStandbySnapshot-race-when-enablin.patch | text/plain | 10.7 KB |
| v1-0001-Fix-LogStandbySnapshot-race-when-enabling-logical.patch | text/plain | 4.6 KB |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Greg Burd | 2026-11-22 16:52:48 | Re: ZSTD TOAST compression, and an extensible compression method encoding |