| From: | Raja Sai pranav <rajasaipranav0(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers(at)lists(dot)postgresql(dot)org |
| Cc: | sulamul(at)gmail(dot)com, robertmhaas(at)gmail(dot)com |
| Subject: | pg_waldump: WAL in pg_wal.old/ inside a tar archive is read as WAL |
| Date: | 2026-11-20 15:22:11 |
| Message-ID: | nh-67f469b8-d10f-4346-8f15-7b31e1d9d168@google.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
While testing the new PG 19 support for reading WAL from tar archives, I
found that member_is_wal_file() in src/bin/pg_waldump/archive_waldump.c
accepts WAL files from directories other than pg_wal/.
The comment says "Skip files in subdirectories other than pg_wal/", but the
check only compares the first six characters:
if (pathlen > XLOG_FNAME_LEN &&
strncmp(pathname, XLOGDIR, strlen(XLOGDIR)) != 0)
return false;
So pg_wal.old/000000010000000000000038 or
pg_wal/somedir/000000010000000000000038 is also taken as WAL. If such a
file comes first in the archive, the real pg_wal/ copy is then skipped as a
duplicate.
This also affects pg_verifybackup. With -Ft and no pg_wal.tar (for example
pg_basebackup -Xfetch), it runs pg_waldump on base.tar.
How I reproduced it on REL_19_STABLE:
1. Put a WAL segment from another cluster into the data directory as
pg_wal.old/000000010000000000000038, using the same name as the segment the
next backup starts in.
2. pg_basebackup -D bk -Ft -Xfetch. base.tar has
pg_wal.old/000000010000000000000038 as member 9 and
pg_wal/000000010000000000000038 as member 1004.
3. pg_waldump -p bk/base.tar -s 0/03800028 -e 0/03800128 exits 0 and prints
a COMMIT record from the other cluster. The real records at that LSN are
CHECKPOINT_REDO, RUNNING_XACTS, CHECKPOINT_ONLINE and BACKUP_END.
4. I zeroed the first records of the real pg_wal/000000010000000000000038
in base.tar. Without pg_wal.old/, pg_verifybackup -F t fails with "WAL
parsing failed for timeline 1". With pg_wal.old/ present, it says "backup
successfully verified".
The attached patch accepts a WAL file only at the top level of the archive
or directly in pg_wal/, and adds a test to
https://www.google.com/url?q=http://001_basic.pl&source=gmail&ust=1795274530565000&sa=E.
The test fails without the code change and passes with it.
With the patch, the corrupted backup above fails verification, and
pg_waldump prints the real records. I checked that pg_waldump output is
unchanged compared to reading the directory for tar files with segments in
order, reversed and shuffled, with pg_wal/ and ./ prefixes, with a
duplicate member, and with gzip, lz4 and zstd compression. The pg_waldump
and pg_verifybackup TAP suites pass on master and REL_19_STABLE.
I think this should be backpatched to 19.
Regards,
Sai Pranav
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-pg_waldump-accept-archive-WAL-members-only-at-top.patch | text/x-patch | 3.3 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Robert Haas | 2026-11-20 15:33:15 | Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e |
| Previous Message | Melih Mutlu | 2026-11-20 15:22:10 | Re: Support "ALTER FOREIGN TABLE ... RENAME CONSTRAINT ..." |