| From: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
|---|---|
| To: | PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org> |
| Cc: | Daniel Gustafsson <daniel(at)yesql(dot)se>, stefangrassl(at)gmx(dot)net |
| Subject: | Fix pg_restore gzip stream validation (bug #19754) |
| Date: | 2026-10-14 07:49:15 |
| Message-ID: | C482E34B-347F-4AD2-ACE1-96142F6088B2@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hi,
Bug #19754 [1] reported an issue with zlib 1.3.2 where gzread() and gzclose() may return incorrect results. The current implementation of pg_restore relies on these two functions, so pg_restore may reject valid dump files or fail to detect truncated gzip streams in some cases. Please read [1] for details if you are interested.
We already have astreamer_gzip, which directly calls zlib's low-level inflate() function to decompress data. So I initially thought about directly reusing astreamer_gzip in compress_gzip.c. However, after some experiments, I realized that its push interface does not fit pg_restore's read interface well. It also reports errors through pg_fatal(), whereas a shared helper should let callers handle errors themselves.
So, I extracted the core decompression logic into a separate gzip.c and used it in both astreamer_gzip and compress_gzip.c.
To reproduce the issue from the bug report, I created a new TAP test, 007_pg_dump_gzip.pl. When built and run with zlib 1.3.2, the test should fail without this patch:
```
# +++ tap check in src/bin/pg_dump +++
t/007_pg_dump_gzip.pl .. 1/?
# Failed test 'restore the intact gzip file whose sizes are on both buffer boundaries'
# at t/007_pg_dump_gzip.pl line 102.
# ---------- command failed ----------
# pg_restore --file /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_restore.sql /Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump
# -------------- stderr --------------
# pg_restore: error: could not close data file "/Users/chaol/Documents/code/postgresql/src/bin/pg_dump/tmp_check/tmp_test_5nHr/gzip_boundary_dump/3875.dat": Undefined error: 0
# ------------------------------------
# Failed test 'fail to restore the truncated gzip data file'
# at t/007_pg_dump_gzip.pl line 120.
```
See the attached patch for details:
* 0001 - Only a TAP test that reproduce bug #19754
* 0002 - The fix plus more test coverage
@Stefan, would you please help verify the fix on your end?
[1] https://www.postgresql.org/message-id/19754-b335a019b76fd89b@postgresql.org
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-Add-a-TAP-test-for-pg_restore-bug-19754.patch | application/octet-stream | 6.2 KB |
| v1-0002-Fix-gzip-stream-validation-in-pg_restore.patch | application/octet-stream | 35.4 KB |
| From | Date | Subject | |
|---|---|---|---|
| Previous Message | Nikhil Sontakke | 2026-10-14 07:47:55 | Re: DDL deparse |