Re: Fix detection of truncated zstd-compressed backups

From: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
To: Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>
Cc: Daniel Gustafsson <daniel(at)yesql(dot)se>, Japin Li <japinli(at)hotmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Osama Abdul Qader <osamaabdulqader(dot)cs(at)gmail(dot)com>
Subject: Re: Fix detection of truncated zstd-compressed backups
Date: 2026-10-12 01:12:03
Message-ID: 489CA674-612D-4DCD-8875-7657B318F71B@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> On Oct 10, 2026, at 19:03, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> wrote:
>
> Hello
>
> An automatic post-commit claude review pointed out that this broke
> restoring directory format dumps with an empty large object, when
> compressed with zstd or lz4:
>
> SELECT lo_create(0);
>
> $ pg_dump -Fd -Z zstd -f dump postgres
> $ pg_restore -f /dev/null dump
> pg_restore: error: could not decompress data: compressed stream is
> incomplete
>
> $ pg_dump -Fd -Z lz4 -f dump postgres
> $ pg_restore -f /dev/null dump
> pg_restore: error: could not read from input file: Input/output error
> pg_restore: error: could not read from input file: Input/output error
>
> The compressed file handles only start a frame on the first write, so
> the blob_NNN.dat.zst/.lz4 file of an empty large object is 0 bytes,
> and the new check treats that as a truncated stream. Custom format and
> gzip are fine.
>
> The attached patch makes the reader accept a file with no input at all
> as a complete stream, a stream that started a frame and then stopped
> is still rejected.
> <v1-0001-pg_dump-Fix-restore-of-empty-zstd-and-LZ4-data-fi.patch>

Thanks for the post-commit verification.

I agree with your analysis and the fix. Initializing frame_finished to true allows the reader to handle an empty input file.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Richard Guo 2026-10-12 01:18:38 ERROR: failed to apply nullingrels to a non-Var
Previous Message Amit Kapila 2026-10-12 00:43:46 Re: Proposal: Conflict log history table for Logical Replication