pg_waldump: WAL in pg_wal.old/ inside a tar archive is read as WAL

From: Raja Sai pranav <rajasaipranav0(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: sulamul(at)gmail(dot)com, robertmhaas(at)gmail(dot)com
Subject: pg_waldump: WAL in pg_wal.old/ inside a tar archive is read as WAL
Date: 2026-11-20 15:22:11
Message-ID: nh-67f469b8-d10f-4346-8f15-7b31e1d9d168@google.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

While testing the new PG 19 support for reading WAL from tar archives, I
found that member_is_wal_file() in src/bin/pg_waldump/archive_waldump.c
accepts WAL files from directories other than pg_wal/.

The comment says "Skip files in subdirectories other than pg_wal/", but the
check only compares the first six characters:

if (pathlen > XLOG_FNAME_LEN &&
strncmp(pathname, XLOGDIR, strlen(XLOGDIR)) != 0)
return false;

So pg_wal.old/000000010000000000000038 or
pg_wal/somedir/000000010000000000000038 is also taken as WAL. If such a
file comes first in the archive, the real pg_wal/ copy is then skipped as a
duplicate.

This also affects pg_verifybackup. With -Ft and no pg_wal.tar (for example
pg_basebackup -Xfetch), it runs pg_waldump on base.tar.

How I reproduced it on REL_19_STABLE:

1. Put a WAL segment from another cluster into the data directory as
pg_wal.old/000000010000000000000038, using the same name as the segment the
next backup starts in.
2. pg_basebackup -D bk -Ft -Xfetch. base.tar has
pg_wal.old/000000010000000000000038 as member 9 and
pg_wal/000000010000000000000038 as member 1004.
3. pg_waldump -p bk/base.tar -s 0/03800028 -e 0/03800128 exits 0 and prints
a COMMIT record from the other cluster. The real records at that LSN are
CHECKPOINT_REDO, RUNNING_XACTS, CHECKPOINT_ONLINE and BACKUP_END.
4. I zeroed the first records of the real pg_wal/000000010000000000000038
in base.tar. Without pg_wal.old/, pg_verifybackup -F t fails with "WAL
parsing failed for timeline 1". With pg_wal.old/ present, it says "backup
successfully verified".

The attached patch accepts a WAL file only at the top level of the archive
or directly in pg_wal/, and adds a test to
https://www.google.com/url?q=http://001_basic.pl&source=gmail&ust=1795274530565000&sa=E.
The test fails without the code change and passes with it.

With the patch, the corrupted backup above fails verification, and
pg_waldump prints the real records. I checked that pg_waldump output is
unchanged compared to reading the directory for tar files with segments in
order, reversed and shuffled, with pg_wal/ and ./ prefixes, with a
duplicate member, and with gzip, lz4 and zstd compression. The pg_waldump
and pg_verifybackup TAP suites pass on master and REL_19_STABLE.

I think this should be backpatched to 19.

Regards,
Sai Pranav

Attachment Content-Type Size
v1-0001-pg_waldump-accept-archive-WAL-members-only-at-top.patch text/x-patch 3.3 KB

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Haas 2026-11-20 15:33:15 Re: O(N^3) planning time in choose_plan_name() since 8c49a484e8e
Previous Message Melih Mutlu 2026-11-20 15:22:10 Re: Support "ALTER FOREIGN TABLE ... RENAME CONSTRAINT ..."