Re: backup manifests

From: Stephen Frost <sfrost(at)snowman(dot)net>
To: Andres Freund <andres(at)anarazel(dot)de>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, David Steele <david(at)pgmasters(dot)net>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>, Suraj Kharage <suraj(dot)kharage(at)enterprisedb(dot)com>, tushar <tushar(dot)ahuja(at)enterprisedb(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Tels <nospam-pg-abuse(at)bloodgate(dot)com>, Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>, Jeevan Chalke <jeevan(dot)chalke(at)enterprisedb(dot)com>, vignesh C <vignesh21(at)gmail(dot)com>
Subject: Re: backup manifests
Date: 2020-03-27 21:07:42
Message-ID: 20200327210742.GJ13712@tamriel.snowman.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Greetings,

* Andres Freund (andres(at)anarazel(dot)de) wrote:
> On 2020-03-27 14:34:19 -0400, Robert Haas wrote:
> > I think #2 is an interesting idea and could possibly reduce the danger
> > of user confusion on this point considerably - because, let's face it,
> > not everyone is going to read the documentation. However, I'm having a
> > hard time figuring out exactly what we'd print. Right now on success,
> > unless you specify -q, you get:
> >
> > [rhaas ~]$ pg_validatebackup ~/pgslave
> > backup successfully verified
> >
> > But it feels strange and possibly confusing to me to print something like:
> >
> > [rhaas ~]$ pg_validatebackup ~/pgslave
> > backup successfully verified (except for pg_wal)
>
> You could print something like:
> WAL necessary to restore this base backup can be validated with:
>
> pg_waldump -p ~/pgslave -t tl -s backup_start_location -e backup_end_loc > /dev/null && echo true
>
> Obviously that specific invocation sucks, but it'd not be hard to add an
> option to waldump to not output anything.

Interesting idea to use pg_waldump.

I had suggested up-thread, and I'm still fine with, having
pg_validatebackup scan the WAL and check the internal checksums. I'd
prefer an option that uses hashes to check when the user has asked for
hashes with SHA256 or something, but at least scanning the WAL and
making sure it validates its internal checksum (and is actually all
there, which is pretty darn critical) would be enough to say that we're
pretty sure the backup is valid.

Thanks,

Stephen

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2020-03-27 21:30:55 Re: Internal key management system
Previous Message Stephen Frost 2020-03-27 20:57:46 Re: backup manifests