Re: Fix detection of truncated zstd-compressed backups

From: Zsolt Parragi <zsolt(dot)parragi(at)percona(dot)com>
To: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
Cc: Daniel Gustafsson <daniel(at)yesql(dot)se>, Japin Li <japinli(at)hotmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Osama Abdul Qader <osamaabdulqader(dot)cs(at)gmail(dot)com>
Subject: Re: Fix detection of truncated zstd-compressed backups
Date: 2026-10-10 11:03:10
Message-ID: CAN4CZFPbcBD+5Xj8XrDa1KQoV75-aWBiKs8n-zEJmQnBG4D3KA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello

An automatic post-commit claude review pointed out that this broke
restoring directory format dumps with an empty large object, when
compressed with zstd or lz4:

SELECT lo_create(0);

$ pg_dump -Fd -Z zstd -f dump postgres
$ pg_restore -f /dev/null dump
pg_restore: error: could not decompress data: compressed stream is
incomplete

$ pg_dump -Fd -Z lz4 -f dump postgres
$ pg_restore -f /dev/null dump
pg_restore: error: could not read from input file: Input/output error
pg_restore: error: could not read from input file: Input/output error

The compressed file handles only start a frame on the first write, so
the blob_NNN.dat.zst/.lz4 file of an empty large object is 0 bytes,
and the new check treats that as a truncated stream. Custom format and
gzip are fine.

The attached patch makes the reader accept a file with no input at all
as a complete stream, a stream that started a frame and then stopped
is still rejected.

Attachment Content-Type Size
v1-0001-pg_dump-Fix-restore-of-empty-zstd-and-LZ4-data-fi.patch application/octet-stream 4.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Srinath Reddy Sadipiralla 2026-10-10 11:08:58 Fwd: [RFC PATCH v1] On-demand WAL replay: accept connections before crash recovery has applied the WAL
Previous Message ZizhuanLiu X-MAN 2026-10-10 10:54:56 Re: examine_variable ignored CollateExpr