| From: | shihao zhong <zhong950419(at)gmail(dot)com> |
|---|---|
| To: | pgsql-hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
| Cc: | sulamul(at)gmail(dot)com, Andrew Dunstan <andrew(at)dunslane(dot)net> |
| Subject: | [PG19]pg_verifybackup never finishes on a gzip-compressed tar backup |
| Date: | 2026-10-06 01:15:49 |
| Message-ID: | CAGRkXqTcyuog_MzKqQm+am2xMefLFJTSKaRevnfypDWirCFvdA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi hackers,
I used Opus to analyze the new features in PG 19.
It found that pg_verifybackup never finishes on a tar-format
backup taken with pg_basebackup -Ft -z.
I had manually proved that by hand.
On REL_19_STABLE:
pg_basebackup -D /tmp/verify_targz.66809/bk -c fast -Ft -z
pg_verifybackup /tmp/verify_targz.66809/bk
# still running after 20 s, killing pg_waldump
pg_verifybackup: error: WAL parsing failed for timeline 1
pg_verifybackup -n /tmp/verify_targz.66809/bk
backup successfully verified
rewrite pg_wal.tar.gz as a single gzip member
pg_verifybackup /tmp/verify_targz.66809/bk
backup successfully verified
With WAL streaming, pg_basebackup ends the gzip stream before and
after every tar header, so pg_wal.tar.gz is a chain of gzip members.
astreamer_gzip_decompressor_content() loops until avail_in is zero.
After the first member inflate() keeps returning Z_STREAM_END and
takes no more input, so the loop never ends. The loop runs in
pg_waldump, and pg_verifybackup waits for it.
Patches attached. 0001 calls inflateReset() when inflate() returns
Z_STREAM_END, so the next member is read. 0002 adds a --gzip case to
the tar test in 007_wal.pl and is optional.
Thanks,
Shihao
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Handle-concatenated-gzip-members-in-astreamer-gzi.patch | application/octet-stream | 1.3 KB |
| v1-0002-Add-test-for-pg_verifybackup-on-a-gzip-compressed.patch | application/octet-stream | 1.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Chao Li | 2026-10-06 01:20:47 | Re: [PG19]pg_verifybackup never finishes on a gzip-compressed tar backup |
| Previous Message | shihao zhong | 2026-10-06 01:04:37 | Re: [PG19] COPY (query) TO ... (FORMAT json) uses the table's column names |