Re: Online verification of checksums

On Fri, Mar 29, 2019 at 4:30 PM Stephen Frost <sfrost(at)snowman(dot)net> wrote:

> Greetings,
>
> * Magnus Hagander (magnus(at)hagander(dot)net) wrote:
> > On Thu, Mar 28, 2019 at 10:19 PM Tomas Vondra <
> tomas(dot)vondra(at)2ndquadrant(dot)com>
> > wrote:
> >
> > > On Thu, Mar 28, 2019 at 01:11:40PM -0700, Andres Freund wrote:
> > > >Hi,
> > > >
> > > >On 2019-03-28 21:09:22 +0100, Michael Banck wrote:
> > > >> I agree that the current patch might have some corner-cases where it
> > > >> does not guarantee 100% accuracy in online mode, but I hope the
> current
> > > >> version at least has no more false negatives.
> > > >
> > > >False positives are *bad*. We shouldn't integrate code that has them.
> > > >
> > >
> > > Yeah, I agree. I'm a bit puzzled by the reluctance to make the online
> mode
> > > communicate with the server, which would presumably address these
> issues.
> > > Can someone explain why not to do that?
> >
> > I agree that this effort seems better spent on fixing those issues there
> > (of which many are the same), and then re-use that.
>
> This really seems like it depends on which of the options we're talking
> about.. Connecting to the server and asking what the current insert
> point is, so we can check that the LSN isn't completely insane, seems
> reasonable, but at least one option being discussed was to have
> pg_basebackup actually *lock the page* (even if just for I/O..) and then
> re-read it, and having an external tool doing that instead of the
> backend seems like a whole different level to me. That would involve
> having an SQL function for "lock this page against I/O" and then another
> for "unlock this page", wouldn't it?
>

Right.

But what if we just added a flag to the BASE_BACKUP command in the
replication protocol that said "meh, I really just want to verify the
checksums, so please send the data to devnull and only feed me regular
status updates on this connection"?

--
Magnus Hagander
Me: https://www.hagander.net/ <http://www.hagander.net/>
Work: https://www.redpill-linpro.com/ <http://www.redpill-linpro.com/>