From 5c430ad809fd67d81c037baccb5dc23bf9182a0c Mon Sep 17 00:00:00 2001 From: David Steele Date: Sat, 24 Oct 2026 21:28:50 +0000 Subject: Avoid truncated or zero-padded copies of replaced files in backups. Base backups stat each file while scanning the data directory and later send exactly the number of bytes reported by the stat. If the file is shorter when it is read then the copy is zero-padded. This is harmless for relation files since any data written after the stat will be restored from WAL. Some files that are not relation files are written to a temp file and then renamed into place, e.g. postgresql.auto.conf by ALTER SYSTEM and pg_logical/replorigin_checkpoint by every checkpoint. If the rename happens between the stat and the open then the backup contains the new file truncated or zero-padded to the size of the old file. WAL replay does not repair these files. A damaged replorigin_checkpoint fails its CRC check and the restored cluster will not start. A damaged postgresql.auto.conf either fails to parse or, if the truncation falls on a line boundary, silently loses settings. Some contrib modules also write files to the data directory this way. pg_prewarm rewrites autoprewarm.blocks every pg_prewarm.autoprewarm_interval. A truncated autoprewarm.blocks fails to load. The autoprewarm worker then exits and is not restarted so blocks are neither prewarmed nor dumped. pg_stash_advice rewrites pg_stash_advice.tsv every pg_stash_advice.persist_interval when the stashed advice has changed. A truncated pg_stash_advice.tsv may load a truncated advice string without error since the last line is not required to end with a newline. For files that are not relation files, fstat() the opened file and use that stat for the tar header and the backup manifest. Since the rename is atomic the opened file is always complete. Relation files are not replaced this way and are still sent using the earlier stat. Calling fstat() for every file would be simpler, but using the earlier stat for relation files means that data added to relation files after the stat is not copied. WAL replay will restore that data anyway so copying it is wasteful. Back-patch to all supported branches. Before v17, sendFile() does not receive the relfilenumber so on those branches sendDir() should save the result of its existing parse_filename_for_nontemp_relation() call and pass it to sendFile(). That call is only made for database directories so relation files in global/ are not identified and get the stat of the opened file. The copy may then include data written between the earlier stat and the open, which is harmless since WAL replay will overwrite that data, just as it overwrites blocks modified while the file is being copied. --- src/backend/backup/basebackup.c | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/src/backend/backup/basebackup.c b/src/backend/backup/basebackup.c index d7c485dc002..ce8fec60cda 100644 --- a/src/backend/backup/basebackup.c +++ b/src/backend/backup/basebackup.c @@ -1575,6 +1575,9 @@ sendDir(bbsink *sink, const char *path, int basepathlen, bool sizeonly, * If the whole file is to be sent, then incremental_blocks should be NULL, * and num_incremental_blocks can have any value, as it will be ignored. * + * If the file is not a relation file then statbuf is replaced with the stat of + * the file that was opened. + * * Returns true if the file was successfully sent, false if 'missing_ok', * and the file did not exist. */ @@ -1608,6 +1611,21 @@ sendFile(bbsink *sink, const char *readfilename, const char *tarfilename, errmsg("could not open file \"%s\": %m", readfilename))); } + /* + * If this is not a relation file then use the stat of the file that was + * actually opened rather than the caller's earlier stat. Some files, e.g. + * postgresql.auto.conf, are written to a temp file and then renamed in + * place. If the rename happened after the earlier stat then the opened + * file may have a different size and sending the earlier size would + * truncate or zero-pad the file. WAL replay will not repair these files. + * Relation files are not replaced this way and any data added to relation + * files after the earlier stat will be restored by WAL replay. + */ + if (!RelFileNumberIsValid(relfilenumber) && fstat(fd, statbuf) != 0) + ereport(ERROR, + (errcode_for_file_access(), + errmsg("could not stat file \"%s\": %m", readfilename))); + _tarWriteHeader(sink, tarfilename, NULL, statbuf, false); /* -- 2.34.1