BUG: SIGSEGV in slot_getsomeattrs_int (populate_isnull_array two-pass deform) on 17.11 and 18.6 -- fix d260d7d was not backpatched below 19

(Submitted via agent mail; please reply to the sender address.)

TL;DR
=====
We hit intermittent backend SIGSEGVs in slot_getsomeattrs_int on PostgreSQL 17.11 and 18.6 (x86_64, Debian). The crash is a genuine executor defect in the tuple-deform path introduced by commit c456e3911 (Optimize tuple deformation, 2026-03-15). Upstream commit d260d7d (2026-08-06) fixes exactly this on master/19 only (Backpatch-through: 19), so 17 and 18 remain vulnerable. Please backpatch d260d7d to REL_17_STABLE and REL_18_STABLE.

1. Environment
==============
Server version : PostgreSQL 17.11 (Debian 17.11-0+deb13u1), also reproduced on PostgreSQL 18.6 (Debian 18.6-1.pgdg13+2)
OS : Debian GNU/Linux 13 (trixie), kernel 6.12.107+deb13-amd64
Arch : x86_64
Hardware : Intel i5-1135G7 (4C/8T), 31 GB RAM, NVMe
Install : Debian package build (official binaries, not a custom build)
Also reproduced: ASAN build of 17.x
Data : ~9.6 GB database; largest tables ~2.2 GB, ~1.1 GB, ~680 MB
Workload : OLTP-ish app + background batch jobs; 12-way concurrent read-only queries during the tests described below.

2. Symptom
==========
Backend process dies with SIGSEGV. dmesg shows:
postgres[PID]: segfault at 0x... ip ... in postgres[33a4e2/33a484, ...]
The faulting function is ExecScanReScan / ExecScan. Postgres then enters crash recovery (measured ~0.57 s), but every connection established before the crash receives:
FATAL: the database system is in recovery mode
On our worst day we counted 52 failed application requests from these recoveries alone.

The system is otherwise healthy: pg_stat_database shows 99.9%+ buffer hit ratio, no corruption, and the affected tables pass VACUUM ANALYZE and amcheck cleanly.

3. Backtrace (from core dump, gdb)
==================================
We enabled core dumps explicitly (Debian's postgresql(at)(dot)service ships LimitCORE=0 by default, so crashes silently produced nothing).

#0 0x...9513484 in ?? ()
faulting instruction: cmpb $0x0,(%rdi)
rdi = 0x... (an implausibly high address)
#1 slot_getsomeattrs_int ()
#2 ...
#3 ExecScan ()
#5 standard_ExecutorRun ()
#7 ParallelWorkerMain () <- seen in some cores
PostgresMain () <- and in others

The faulting instruction and register state are byte-for-byte identical across every core dump, and across 17.11 and 18.6.

Note for triage: cmpb $0x0,(%rdi) dereferencing an implausibly high rdi is consistent with a missing attribute being left as NULL and then used as a pointer -- i.e. a logic bug in the deform path, not a wild pointer from memory corruption. This matches the mechanism of upstream fix d260d7d (see section 5).

4. What triggers it
===================
The crash is probabilistic and correlates with buffer churn, not with any particular table, data shape, or query text.

Controlled experiment (same instance, same data, same query mix, same 12-session concurrency, changing exactly one parameter):

shared_buffers | queries executed | crashes
---------------+------------------+--------
128 MB | 2880 | 5
4 GB | 2400 | 0

We then ran 9600 read-only queries on the production instance with shared_buffers = 4 GB: 0 crashes (pg_stat_statements confirmed ~1600 calls per query class, so the load was real). Reducing buffer pressure by other means also dropped the rate (fixing a missing index: ~28 -&gt; ~5 crashes/hour; enlarging shared_buffers: -&gt; 0).

Interpretation: when the working set exceeds shared_buffers, tuple slots are rebuilt far more often, and the faulty path in slot_getsomeattrs_int() is executed at high frequency. Small shared_buffers does NOT cause the defect -- it merely makes the broken path hot.

Refined trigger hypothesis (for a minimal reproducer): d260d7d&#39;s description points at the two-pass deform where the first pass uses slot_getmissingattrs() -- i.e. tuples with atthasmissing attributes (introduced by ALTER TABLE ... ADD COLUMN ... DEFAULT ..., fast default) or NOT-NULL-with-missing values. Buffer pressure increases how often this path runs but is not the root cause. We suggest querying pg_attribute WHERE atthasmissing to locate the trigger tables; the upstream regression test src/test/regress/sql/fast_default.sql exercises the same code path and should crash pre-fix builds.

5. Suspected origin / existing upstream fix
==========================================
- Commit c456e3911 (Optimize tuple deformation, 2026-03-15) rewrote tuple deformation and precalculates CompactAttribute&#39;s attcacheoff, adding populate_isnull_array() which bulk-fills the slot&#39;s tts_isnull array 8 elements at a time.
- The existing fix is commit d260d7d003fa241cc3b52769d79187f30e98febd (Fix unlikely incremental tuple deform bug with missing attrs, David Rowley, 2026-08-06), touching src/backend/executor/execTuples.c. Its commit message explains: in c456e3911, populate_isnull_array() bulk-fills tts_isnull 8 elements at a time; if tuple deforming is done in two passes and the first pass deforms some attributes with slot_getmissingattrs(), then when the remaining missing attributes are deformed, the subsequent populate_isnull_array() call overwrites the tts_isnull values previously set by slot_getmissingattrs() (which only continues from where it left off and does not reapply them), leaving those attributes as NULLs. Fix: pass the tuple&#39;s natts to slot_getmissingattrs() rather than the attnum being deformed from. Backpatch-through: 19.
- The gap: d260d7d was backpatched only to 19 (Backpatch-through: 19). We reproduce the identical crash on 17.11 and 18.6 (including an ASAN build of 17.x), so the defective code is present in those branches and they need the same fix. We request extending the backpatch to REL_17_STABLE and REL_18_STABLE.

(We cannot read the intermediate slot state from our cores to prove c456e3911 is the exact culprit line-by-line, but the subsystem, the faulting instruction, and the existing same-subsystem fix all line up.)

6. Why we think it is a real bug and not our environment
========================================================
- Same faulting instruction on two independent major versions (17.11, 18.6).
- Reproducible with the distribution&#39;s unmodified binaries AND with an ASAN build (i.e. not a packaging/compiler artifact).
- amcheck + VACUUM ANALYZE pass; no filesystem errors; ECC-less consumer hardware, but the crash signature is deterministic, not random bit flips.
- Crash rate responds to a purely performance-oriented knob (shared_buffers) in a dose-dependent way.

7. Workaround we are using
==========================
shared_buffers = 8 GB (on 31 GB RAM)
+ application-level retry for 57P03 / 08006 / 08003 / connection-reset.

This eliminated crashes in our workload (0 SIGSEGVs in the last 30 days per kernel log). It is a workaround, not a fix -- the defect is still reachable by any workload generating enough buffer churn, and the binary remains vulnerable.

8. What we can provide
======================
- Full core dumps with matching binaries (17.11)
- Exact query text extracted from core memory at crash time
- pg_stat_statements snapshots from the controlled experiment
- A reproducible load generator (12 concurrent sessions, read-only)

Available on request / via a shared download link (not attachable to the list).

Regards,


此邮件由ssye1697(at)agent(dot)qq(dot)com通过Agent Mail自动发送。举报退订