Re: Online verification of checksums

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Tomas Vondra <tomas(dot)vondra(at)2ndquadrant(dot)com>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Stephen Frost <sfrost(at)snowman(dot)net>, Michael Banck <michael(dot)banck(at)credativ(dot)de>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: Online verification of checksums
Date: 2019-03-06 02:36:40
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Mar 05, 2019 at 02:08:03PM +0100, Tomas Vondra wrote:
> Based on quickly skimming that thread the main issue seems to be
> deciding which files in the data directory are expected to have
> checksums. Which is a valid issue, of course, but I was expecting
> something about partial read/writes etc.

I remember complaining about partial write handling as well for the
base backup checks... There should be an email about it on the list,
cannot find it now ;p

> My understanding is that:
> (a) The checksum verification should not generate false positives (same
> as for basebackup).
> (b) The partial reads do emit warnings, which might be considered false
> positives I guess. Which is why I'm arguing for changing it to do the
> same thing basebackup does, i.e. ignore this.

Well, at least that's consistent... Argh, I really think that we
ought to make the failures reported harder because that's easier to
detect within a tool and some deployments set log_min_messages >
WARNING so checksum failures would just be lost. For base backups we
don't care much about that as files are just blindly copied so they
could have torn pages, which is fine as that's fixed at replay. Now
we are talking about a set of tools which could have reliable
detection mechanisms for those problems.

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2019-03-06 02:42:21 Re: Online verification of checksums
Previous Message Amit Langote 2019-03-06 02:34:27 Re: Update does not move row across foreign partitions in v11