| From: | Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> |
|---|---|
| To: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
| Cc: | Daniel Gustafsson <daniel(at)yesql(dot)se>, Japin Li <japinli(at)hotmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Osama Abdul Qader <osamaabdulqader(dot)cs(at)gmail(dot)com> |
| Subject: | Re: Fix detection of truncated zstd-compressed backups |
| Date: | 2026-10-10 11:03:10 |
| Message-ID: | CAN4CZFPbcBD+5Xj8XrDa1KQoV75-aWBiKs8n-zEJmQnBG4D3KA@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hello
An automatic post-commit claude review pointed out that this broke
restoring directory format dumps with an empty large object, when
compressed with zstd or lz4:
SELECT lo_create(0);
$ pg_dump -Fd -Z zstd -f dump postgres
$ pg_restore -f /dev/null dump
pg_restore: error: could not decompress data: compressed stream is
incomplete
$ pg_dump -Fd -Z lz4 -f dump postgres
$ pg_restore -f /dev/null dump
pg_restore: error: could not read from input file: Input/output error
pg_restore: error: could not read from input file: Input/output error
The compressed file handles only start a frame on the first write, so
the blob_NNN.dat.zst/.lz4 file of an empty large object is 0 bytes,
and the new check treats that as a truncated stream. Custom format and
gzip are fine.
The attached patch makes the reader accept a file with no input at all
as a complete stream, a stream that started a frame and then stopped
is still rejected.
| Attachment | Content-Type | Size |
|---|---|---|
| v1-0001-pg_dump-Fix-restore-of-empty-zstd-and-LZ4-data-fi.patch | application/octet-stream | 4.0 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Srinath Reddy Sadipiralla | 2026-10-10 11:08:58 | Fwd: [RFC PATCH v1] On-demand WAL replay: accept connections before crash recovery has applied the WAL |
| Previous Message | ZizhuanLiu X-MAN | 2026-10-10 10:54:56 | Re: examine_variable ignored CollateExpr |