| From: | shihao zhong <zhong950419(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | Robert Haas <robertmhaas(at)gmail(dot)com>, Melanie Plageman <melanieplageman(at)gmail(dot)com>, Andres Freund <andres(at)anarazel(dot)de> |
| Subject: | Incremental backup keeps VM bits cleared by VACUUM truncation |
| Date: | 2026-10-21 02:39:56 |
| Message-ID: | CAGRkXqSnghF+wbyP-PdfjVLfbejJG3cvZd9hN-gxsvCWj2F64Q@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
With data_checksums and wal_log_hints off, a cluster restored from an
incremental backup can keep stale VM bits after VACUUM truncates a
table. When the table grows again, index-only scans return deleted and
rolled back rows.
The attached script shows it. One VACUUM empties the end of a table but
skips the truncation on a lock conflict. After the full backup, a later
VACUUM truncates it. On 17 and 18 the results stay wrong even after
another VACUUM. On 19 and master, VACUUM fixes them.
visibilitymap_prepare_truncate() clears bits in the last VM page, but
writes that page to WAL only if XLogHintBitIsNeeded(). So the WAL
summary never sees the change, and the incremental backup skips the
page.
0001 always writes the page to WAL, as Andres suggested in [1]. Clusters
with checksums on already do this. 0002 adds a test that fails without
0001.
0001 and 0002 apply to master, 19 and 18. 17 needs the attached nocfbot
file.
The standby PANIC fix in [2] touches the same code path but does
not cover this, so I am reporting it separately.
Command to run attached sh
sh incr-vm-truncate.sh /pgxxx/bin
[1]
https://postgr.es/m/t7xd6gjxdxpzdzr2uamfbyi5rlkdsikiywmtmgwrwcsr77xyw7@i3qhbzy5rpzx
[2]
https://postgr.es/m/CAL4mQLAp562c1rCgg2Dqx6TBSdOk6vOvaLFjwFG9E6k4uwvJJw@mail.gmail.com
Thanks for taking a look,
Shihao
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Always-WAL-log-the-VM-page-changed-by-truncation.patch | application/octet-stream | 1.9 KB |
| v1-0002-Test-incremental-backup-after-a-visibility-map-tr.patch | application/octet-stream | 3.2 KB |
| nocfbot-v1-REL_17_STABLE.patch | application/octet-stream | 5.0 KB |
| incr-vm-truncate.sh | text/x-sh | 2.1 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | David Rowley | 2026-10-21 02:50:08 | Re: [PATCH] Remove useless ORDER BY from scalar subqueries |
| Previous Message | Chao Li | 2026-10-21 01:50:23 | Re: tablecmds: Deadlock caused by Attach Partition |