| From: | Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> |
|---|---|
| To: | Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> |
| Cc: | Daniel Gustafsson <daniel(at)yesql(dot)se>, Japin Li <japinli(at)hotmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Osama Abdul Qader <osamaabdulqader(dot)cs(at)gmail(dot)com> |
| Subject: | Re: Fix detection of truncated zstd-compressed backups |
| Date: | 2026-10-12 01:12:03 |
| Message-ID: | 489CA674-612D-4DCD-8875-7657B318F71B@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
> On Oct 10, 2026, at 19:03, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> wrote:
>
> Hello
>
> An automatic post-commit claude review pointed out that this broke
> restoring directory format dumps with an empty large object, when
> compressed with zstd or lz4:
>
> SELECT lo_create(0);
>
> $ pg_dump -Fd -Z zstd -f dump postgres
> $ pg_restore -f /dev/null dump
> pg_restore: error: could not decompress data: compressed stream is
> incomplete
>
> $ pg_dump -Fd -Z lz4 -f dump postgres
> $ pg_restore -f /dev/null dump
> pg_restore: error: could not read from input file: Input/output error
> pg_restore: error: could not read from input file: Input/output error
>
> The compressed file handles only start a frame on the first write, so
> the blob_NNN.dat.zst/.lz4 file of an empty large object is 0 bytes,
> and the new check treats that as a truncated stream. Custom format and
> gzip are fine.
>
> The attached patch makes the reader accept a file with no input at all
> as a complete stream, a stream that started a frame and then stopped
> is still rejected.
> <v1-0001-pg_dump-Fix-restore-of-empty-zstd-and-LZ4-data-fi.patch>
Thanks for the post-commit verification.
I agree with your analysis and the fix. Initializing frame_finished to true allows the reader to handle an empty input file.
Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Richard Guo | 2026-10-12 01:18:38 | ERROR: failed to apply nullingrels to a non-Var |
| Previous Message | Amit Kapila | 2026-10-12 00:43:46 | Re: Proposal: Conflict log history table for Logical Replication |