Re: backup manifests

On Fri, Apr 03, 2020 at 03:22:23PM -0400, Robert Haas wrote:
> I've pushed all the patches.

I didn't manage to look at this in advance but have some doc fixes.

word-diff:

diff --git a/doc/src/sgml/protocol.sgml b/doc/src/sgml/protocol.sgml
index 536de9a698..d84afb7b18 100644
--- a/doc/src/sgml/protocol.sgml
+++ b/doc/src/sgml/protocol.sgml
@@ -2586,7 +2586,7 @@ The commands accepted in replication mode are:
and sent along with the backup. The manifest is a list of every
file present in the backup with the exception of any WAL files that
may be included. It also stores the size, last modification time, and
[-an optional-]{+optionally a+} checksum for each file.
A value of <literal>force-escape</literal> forces all filenames
to be hex-encoded; otherwise, this type of encoding is performed only
for files whose names are non-UTF8 octet sequences.
@@ -2602,7 +2602,7 @@ The commands accepted in replication mode are:
<term><literal>MANIFEST_CHECKSUMS</literal></term>
<listitem>
<para>
Specifies the {+checksum+} algorithm that should be applied to each file included
in the backup manifest. Currently, the available
algorithms are <literal>NONE</literal>, <literal>CRC32C</literal>,
<literal>SHA224</literal>, <literal>SHA256</literal>,
diff --git a/doc/src/sgml/ref/pg_basebackup.sgml b/doc/src/sgml/ref/pg_basebackup.sgml
index c778e061f3..922688e227 100644
--- a/doc/src/sgml/ref/pg_basebackup.sgml
+++ b/doc/src/sgml/ref/pg_basebackup.sgml
@@ -604,7 +604,7 @@ PostgreSQL documentation
not contain any checksums. Otherwise, it will contain a checksum
of each file in the backup using the specified algorithm. In addition,
the manifest will always contain a <literal>SHA256</literal>
checksum of its own [-contents.-]{+content.+} The <literal>SHA</literal> algorithms
are significantly more CPU-intensive than <literal>CRC32C</literal>,
so selecting one of them may increase the time required to complete
the backup.
@@ -614,7 +614,7 @@ PostgreSQL documentation
of each file for users who wish to verify that the backup has not been
tampered with, while the CRC32C algorithm provides a checksum which is
much faster to calculate and good at catching errors due to accidental
changes but is not resistant to [-targeted-]{+malicious+} modifications. Note that, to
be useful against an adversary who has access to the backup, the backup
manifest would need to be stored securely elsewhere or otherwise
verified not to have been modified since the backup was taken.
diff --git a/doc/src/sgml/ref/pg_validatebackup.sgml b/doc/src/sgml/ref/pg_validatebackup.sgml
index 19888dc196..748ac439a6 100644
--- a/doc/src/sgml/ref/pg_validatebackup.sgml
+++ b/doc/src/sgml/ref/pg_validatebackup.sgml
@@ -41,12 +41,12 @@ PostgreSQL documentation
</para>

<para>
It is important to note that[-that-] the validation which is performed by
<application>pg_validatebackup</application> does not and [-can not-]{+cannot+} include
every check which will be performed by a running server when attempting
to make use of the backup. Even if you use this tool, you should still
perform test restores and verify that the resulting databases work as
expected and that they[-appear to-] contain the correct data. However,
<application>pg_validatebackup</application> can detect many problems
that commonly occur due to storage problems or user error.
</para>
@@ -73,7 +73,7 @@ PostgreSQL documentation
a <literal>backup_manifest</literal> file in the target directory or
about anything inside <literal>pg_wal</literal>, even though these
files won't be listed in the backup manifest. Only files are checked;
the presence or absence [-or-]{+of+} directories is not verified, except
indirectly: if a directory is missing, any files it should have contained
will necessarily also be missing.
</para>
@@ -84,7 +84,7 @@ PostgreSQL documentation
for any files for which the computed checksum does not match the
checksum stored in the manifest. This step is not performed for any files
which produced errors in the previous step, since they are already known
to have problems. [-Also, files-]{+Files+} which were ignored in the previous step are
also ignored in this step.
</para>

@@ -123,7 +123,7 @@ PostgreSQL documentation
<title>Options</title>

<para>
The following command-line options control the [-behavior.-]{+behavior of this program.+}

<variablelist>
<varlistentry>
diff --git a/src/backend/replication/basebackup.c b/src/backend/replication/basebackup.c
index 3b18e733cd..aa72a6ff10 100644
--- a/src/backend/replication/basebackup.c
+++ b/src/backend/replication/basebackup.c
@@ -1148,7 +1148,7 @@ AddFileToManifest(manifest_info *manifest, const char *spcoid,
}

/*
* Each file's entry [-need-]{+needs+} to be separated from any entry that follows by a
* comma, but there's no comma before the first one or after the last one.
* To make that work, adding a file to the manifest starts by terminating
* the most recently added line, with a comma if appropriate, but does not

--
Justin