Re: pgsql: Add TAP tests for pg_verify_checksums

From: Michael Banck <michael(dot)banck(at)credativ(dot)de>
To: Michael Paquier <michael(at)paquier(dot)xyz>, Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andres Freund <andres(at)anarazel(dot)de>, Postgres hackers <pgsql-hackers(at)postgresql(dot)org>, Daniel Gustafsson <daniel(at)yesql(dot)se>, Magnus Hagander <magnus(at)hagander(dot)net>
Subject: Re: pgsql: Add TAP tests for pg_verify_checksums
Date: 2018-11-27 13:09:05
Message-ID: 1543324145.17247.6.camel@credativ.de
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-committers pgsql-hackers

Hi,

Am Freitag, den 19.10.2018, 22:50 +0900 schrieb Michael Paquier:
> On Wed, Oct 17, 2018 at 05:30:05PM -0400, Andrew Dunstan wrote:
> Thanks. This is now committed after some tweaks to the comments, a bit
> earlier than I thought first.

I found an issue with this [d55241af7, "Use whitelist to choose files
scanned with pg_verify_checksums"] commit, namely, it makes
pg_verify_checksums no longer scan non-default tablespaces. So if you
have all of your data in tablespaces, it will more-or-less immediately
return with success.

I've extended the test suite to induce corruption in a table located in
a non-default tablespace, see the attached patch.

If fails like this, i.e. does not detect the corruption:

t/002_actions.pl .. 14/42
# Failed test 'fails with corrupted data in non-default tablespace status (got 0 vs expected 1)'
# at t/002_actions.pl line 87.

# Failed test 'fails with corrupted data in non-default tablespace stdout /(?^:Bad checksums:.*1)/'
# at t/002_actions.pl line 87.
# 'Checksum scan completed
# Data checksum version: 1
# Files scanned: 1102
# Blocks scanned: 2861
# Bad checksums: 0
# '
# doesn't match '(?^:Bad checksums:.*1)'

# Failed test 'fails with corrupted data in non-default tablespace stderr /(?^:checksum verification failed)/'
# at t/002_actions.pl line 87.
# ''
# doesn't match '(?^:checksum verification failed)'
# Looks like you failed 3 tests of 42.
t/002_actions.pl .. Dubious, test returned 3 (wstat 768, 0x300)
Failed 3/42 subtests

The problem is that "PG_12_201811201" is not considered a valid relation
file by isRelFileName(), so it skips it and the rest of the tablespace.

I had a quick look at fixing this but did not manage to immediately come
up with a solution, so posting here for now.

Michael

--
Michael Banck
Projektleiter / Senior Berater
Tel.: +49 2166 9901-171
Fax: +49 2166 9901-100
Email: michael(dot)banck(at)credativ(dot)de

credativ GmbH, HRB Mönchengladbach 12080
USt-ID-Nummer: DE204566209
Trompeterallee 108, 41189 Mönchengladbach
Geschäftsführung: Dr. Michael Meskes, Jörg Folz, Sascha Heuer

Unser Umgang mit personenbezogenen Daten unterliegt
folgenden Bestimmungen: https://www.credativ.de/datenschutz

Attachment Content-Type Size
pg_verify_checksums_non-default-tablespace.patch text/x-patch 4.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2018-11-27 13:12:04 Re: [RFC] Removing "magic" oids
Previous Message David Rowley 2018-11-27 12:55:13 Re: Inadequate executor locking of indexes

Browse pgsql-committers by date

  From Date Subject
Next Message Michael Paquier 2018-11-27 13:52:21 Re: pgsql: Add TAP tests for pg_verify_checksums
Previous Message Peter Eisentraut 2018-11-27 12:45:20 Re: pgsql: Integrate recovery.conf into postgresql.conf