pgsql: Avoid truncated or zero-padded copies of replaced files in backu

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: pgsql-committers(at)lists(dot)postgresql(dot)org
Subject: pgsql: Avoid truncated or zero-padded copies of replaced files in backu
Date: 2026-10-27 04:56:38
Message-ID: E1xLZEk-00000000vnw-0HrN@gemulon.postgresql.org
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-committers

Avoid truncated or zero-padded copies of replaced files in backups

Base backups call stat() on each file while scanning the data directory,
and later send exactly that many bytes. If the file is replaced by for
example a concurrent rename(), the backup contains the new file
truncated or zero-padded to the size of the old one. This is harmless
for relation files as WAL replay repairs them. However, it causes harm
for non-relation files manipulated as temporary file renamed in place.

Here is a list of non-relation files that use a temporary file followed
by a final rename() (I may be missing some):
* postgresql.auto.conf
* pg_logical/replorigin_checkpoint (a damaged copy fails its CRC check
and prevents startup)
* pg_prewarm's autoprewarm.blocks
* pg_stash_advice.tsv.

When sending files that are not relation files, fstat() is called again
on the opened file and the refreshed stats are used for the tar header
and the backup manifest. Since the rename
is atomic, the opened file is always complete. Relation files keep using
the earlier stat, to avoid copying data that WAL replay would restore
anyway (because that's cheaper).

Back-patch down to v15. v14 is going to be EOL soon, so I have decided
to leave it out of the picture (risk/reward thing).

Author: David Steele <david(at)pgbackrest(dot)org>
Reviewed-by: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
Reviewed-by: Michael Paquier <michael(at)paquier(dot)xyz>
Reviewed-by: Shihao Zhong <zhong950419(at)gmail(dot)com>
Discussion: https://postgr.es/m/CAGRkXqS_On-pLgcv6xjH8mvGhdcPmfWtythW3r8pYKUG+hsiuQ@mail.gmail.com
Backpatch-through: 15

Branch
------
master

Details
-------
https://git.postgresql.org/pg/commitdiff/51d7bf5edcaf4f8a95bb10f96b7af76d00a2f83f

Modified Files
--------------
src/backend/backup/basebackup.c | 33 ++++++++++++++++++++++++---------
1 file changed, 24 insertions(+), 9 deletions(-)

Browse pgsql-committers by date

  From Date Subject
Next Message Michael Paquier 2026-10-27 06:09:45 pgsql: Add custom extension states to QueryDesc
Previous Message Michael Paquier 2026-10-27 03:25:12 pgsql: Count the last block of the validation scan in a concurrent inde