Re: Substituting Checksum Algorithm (was: Enabling Checksums)

On 2013-04-30 18:39:09 -0400, Greg Smith wrote:
> The WAL logging of hint bits is where the scary stuff to me for this feature
> has always been at. My gut feel is that doing that needed to start being
> available as an option anyway. Just this month we've had two customer
> issues pop up where we had to look for block differences between a master
> and a standby. The security update forced some normal update stragglers to
> where they now have the 9.1.6 index corruption fix, and we're looking for
> cases where standby indexes might have been corrupted by it. In this case
> the comparisons can just avoid anything but indexes, so hint bits are
> thankfully not involved.
>
> But having false positives pop out of comparing a master and standby due to
> hint bits makes this sort of process much harder in general. Being able to
> turn checksums on, and then compare more things between master and standby
> without expecting any block differences, that will make both routine quality
> auditing and forensics of broken clusters so much easier.

I don't think the current implementation helps you with that. We only
log the first hint bit set after a checkpoint, you will still get
inconsistent bits set after that. So you might have some fewer
inconsistencies but not enough to weed them out manually or such.
c.f. MarkBufferDirtyHint() and XLogSaveBufferForHint().

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services