Re: pgsql: Add TAP tests for pg_verify_checksums

Greetings,

On Sat, Oct 20, 2018 at 01:11 Andres Freund <andres(at)anarazel(dot)de> wrote:

> Hi,
>
> On 2018-10-20 01:07:43 -0400, Stephen Frost wrote:
> > I have to say that I can’t recall hearing much in the way of complaints
> > about pg_basebackup copying all the random cstore files
>
> Why would somebody complain about that? It's usually desirable.

Even though they’re as likely as not to be invalid or corrupted..? Maybe
things have moved forward here, I know there’s been discussion about it,
but last I heard those files weren’t WAL’d and therefore the result of
copying them from a running server was indeterminate. Yes, sometimes
they’ll be fine, but you could say the same about regular PG relations too
and yet we certainly wouldn’t be accepting of that. It certainly seems
reasonable that people would complain about pg_basebackup misperforming
when a backup that it did results in an invalid restore, though it tends to
be a lot rarer to get complaints about partial failures like a corrupt or
partial file being copied during a backup- but then that’s part of why we
stress so much about trying to make sure we don’t do that as it can be hard
to detect.

People certainly did complain about unlogged tables being backed up and
that was just because they took up space in the backup and time on the
backup and restore just to be nuked when the server is started.

> or the new checksum validation logic complaining about them, and such
> > when doing backups and I wonder if that is because people simply don’t
> > use the two together much, making me wonder how much of an issue this
> > really is or would be with the account-for-everything approach I’ve
> > been advocating for.
>
> I mean obviously pg_verify_checksum simply hasn't been actually tested
> much with plain postgres without extensions, given the all weaknesses
> identified in this thread.

No, it hasn’t, but pg_basebackup has been around quite a while and has
always copied everything, as best as I can recall anyway.

Thanks,

Stephen

>