Re: 16-bit page checksums for 9.2

On Sat, Dec 24, 2011 at 8:06 PM, Greg Stark <stark(at)mit(dot)edu> wrote:
> On Sat, Dec 24, 2011 at 4:06 PM, Simon Riggs <simon(at)2ndquadrant(dot)com> wrote:
>> Checksums merely detect a problem, whereas FPWs correct a problem if
>> it happens, but only in crash situations.
>>
>> So this does nothing to remove the need for FPWs, though checksum
>> detection could be used for double write buffers also.
>
> This is missing the point. If you have a torn page on a page that is
> only dirty due to hint bits then the checksum will show a spurious
> checksum failure. It will "detect" a problem that isn't there.

It will detect a problem that *is* there, but one you are classifying
it as a non-problem because it is a correctable or acceptable bit
error. Given that acceptable bit errors on hints cover no more than 1%
of a block, the great likelihood is that the bit error is unacceptable
in any case, so false positives page errors are in fact very rare.

Any bit error is an indicator of problems on the external device, so
many would regard any bit error as unacceptable.

> The problem is that there is no WAL indicating the hint bit change.
> And if the torn page includes the new checksum but not the new hint
> bit or vice versa it will be a checksum mismatch.
>
> The strategy discussed in the past was moving all the hint bits to a
> common area and skipping them in the checksum. No amount of double
> writing or buffering or locking will avoid this problem.

I completely agree we should do this, but we are unable to do it now,
so this patch is a stop-gap and provides a much requested feature
*now*.

In the future, we will be able to tell the difference between an
acceptable and an unacceptable bit error. Right now, all we have is
the ability to detect a bit error and as I point out above that is 99%
of the problem solves, at least.

--
 Simon Riggs                   http://www.2ndQuadrant.com/
 PostgreSQL Development, 24x7 Support, Training & Services