Re: documenting the backup manifest file format

On 4/15/20 6:43 PM, Jehan-Guillaume de Rorthais wrote:
> On Wed, 15 Apr 2020 12:03:28 -0400
> Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
>> On Wed, Apr 15, 2020 at 11:23 AM Jehan-Guillaume de Rorthais
>> <jgdr(at)dalibo(dot)com> wrote:
>>> But for backup_manifest, it's kind of shame we have to check the checksum
>>> against an transformed version of the file. Did you consider creating eg. a
>>> separate backup_manifest.sha256 file?
>>>
>>> I'm very sorry in advance if this has been discussed previously.
>>
>> It was briefly mentioned in the original (lengthy) discussion, but I
>> think there was one vote in favor and two votes against or something
>> like that, so it didn't go anywhere.
>
> Argh.
>
>> I didn't realize that there were handy command-line tools for manipulating
>> json like that, or I probably would have considered that idea more strongly.
>
> That was indeed a lengthy thread with various details discussed. I'm sorry I
> didn't catch the ball back then.

One of the reasons to use JSON was to be able to use command line tools
like jq to do tasks (I use it myself). But I think only the
pg_verifybackup tool should be used to verify the internal checksum.

Two thoughts:

1) You can always generate an external checksum when you generate the
backup if you want to do your own verification without running
pg_verifybackup.

2) Perhaps it would be good if the pg_verifybackup command had a
--verify-manifest-checksum option (or something) to check that the
manifest file looks valid without checking any files. That's not going
to happen for PG13, but it's possible for PG14.

Regards,
--
-David
david(at)pgmasters(dot)net