| From: | Michael Paquier <michael(at)paquier(dot)xyz> |
|---|---|
| To: | pgsql-committers(at)lists(dot)postgresql(dot)org |
| Subject: | pgsql: Avoid truncated or zero-padded copies of replaced files in backu |
| Date: | 2026-10-27 04:56:38 |
| Message-ID: | E1xLZEk-00000000vnw-0HrN@gemulon.postgresql.org |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-committers |
Avoid truncated or zero-padded copies of replaced files in backups
Base backups call stat() on each file while scanning the data directory,
and later send exactly that many bytes. If the file is replaced by for
example a concurrent rename(), the backup contains the new file
truncated or zero-padded to the size of the old one. This is harmless
for relation files as WAL replay repairs them. However, it causes harm
for non-relation files manipulated as temporary file renamed in place.
Here is a list of non-relation files that use a temporary file followed
by a final rename() (I may be missing some):
* postgresql.auto.conf
* pg_logical/replorigin_checkpoint (a damaged copy fails its CRC check
and prevents startup)
* pg_prewarm's autoprewarm.blocks
* pg_stash_advice.tsv.
When sending files that are not relation files, fstat() is called again
on the opened file and the refreshed stats are used for the tar header
and the backup manifest. Since the rename
is atomic, the opened file is always complete. Relation files keep using
the earlier stat, to avoid copying data that WAL replay would restore
anyway (because that's cheaper).
Back-patch down to v15. v14 is going to be EOL soon, so I have decided
to leave it out of the picture (risk/reward thing).
Author: David Steele <david(at)pgbackrest(dot)org>
Reviewed-by: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
Reviewed-by: Shihao Zhong <zhong950419(at)gmail(dot)com>
Discussion: https://postgr.es/m/CAGRkXqS_On-pLgcv6xjH8mvGhdcPmfWtythW3r8pYKUG+hsiuQ@mail.gmail.com
Backpatch-through: 15
Branch
------
master
Details
-------
https://git.postgresql.org/pg/commitdiff/51d7bf5edcaf4f8a95bb10f96b7af76d00a2f83f
Modified Files
--------------
src/backend/backup/basebackup.c | 33 ++++++++++++++++++++++++---------
1 file changed, 24 insertions(+), 9 deletions(-)
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Michael Paquier | 2026-10-27 06:09:45 | pgsql: Add custom extension states to QueryDesc |
| Previous Message | Michael Paquier | 2026-10-27 03:25:12 | pgsql: Count the last block of the validation scan in a concurrent inde |