Re: Fix detection of truncated zstd-compressed backups

> On Aug 8, 2026, at 08:00, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> wrote:
>
>
>
>> On Aug 8, 2026, at 03:42, Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com> wrote:
>>
>> Hello!
>>
>> Isn't this a generic problem, also present in gzip/lz4 compressed backups?
>
> Maybe, but I haven’t checked them yet. I will do that next week.

I just verified gzip has the same problem using a similar script:
```
workdir=$(mktemp -d /tmp/gzip-trunc.XXXXXX)
mkdir "$workdir/truncated"
dd if=/dev/zero of="$workdir/base.tar" bs=1024 count=2
gzip -c "$workdir/base.tar" > "$workdir/base.tar.gz"
size=$(stat -f %z "$workdir/base.tar.gz")
dd if="$workdir/base.tar.gz" of="$workdir/truncated/base.tar.gz" bs=1 count=$((size - 1))

manifest_prefix=$'{"PostgreSQL-Backup-Manifest-Version": 1,\n "Files": [],\n "WAL-Ranges": [],\n'
printf '%s' "$manifest_prefix" > "$workdir/manifest-prefix"
manifest_checksum=$(shasum -a 256 "$workdir/manifest-prefix" | awk '{print $1}')
printf '%s"Manifest-Checksum": "%s"}\n' "$manifest_prefix" "$manifest_checksum" > "$workdir/truncated/backup_manifest"

gzip -t "$workdir/truncated/base.tar.gz"
pg_verifybackup -F t -s "$workdir/truncated"
```

gzip fails, while pg_verifybackup succeeds.

Looking at the code, astreamer_gzip_decompressor_content() stops processing when it has consumed the input buffer. However, the inflate() doc, see [2] and [3], says that only Z_STREAM_END indicates that decompression has completed. My debugging also shows that, when reading the truncated gzip file, inflate() returns Z_OK, while it returns Z_STREAM_END for a valid gzip file. Therefore, this patch records whether inflate() has returned Z_STREAM_END and checks that state in astreamer_gzip_decompressor_finalize().

Unlike zstd, we don't need to call inflate() again in astreamer_gzip_decompressor_finalize(). Once all input has been consumed without Z_STREAM_END, calling inflate() with no input cannot complete the stream.

See attached 0002 for the fix of gzip streamer. I will check the lz4 streamer next.

Best regards,
--
Chao Li (Evan)
HighGo Software Co., Ltd.
https://www.highgo.com/