Re: Block-level CRC checks

Robert Haas wrote:
> On Tue, Dec 1, 2009 at 8:30 AM, Heikki Linnakangas
> <heikki(dot)linnakangas(at)enterprisedb(dot)com> wrote:
>> Bruce Momjian wrote:
>>> What might be interesting is to report CRC mismatches if the database
>>> was shut down cleanly previously; I think in those cases we shouldn't
>>> have torn pages.
>> Unfortunately that's not true. You can crash, leading to a torn page,
>> and then start up the database and shut it down cleanly. The torn page
>> is still there, even though the last shutdown was a clean one.
>
> Thinking through this, as I understand it, in order to prevent this
> problem, you'd need to be able to predict at recovery time which pages
> might have been torn by the unclean shutdown. In order to do that,
> you'd need to know which pages were waiting to be written to disk at
> the time of the shutdown. For ordinary page modifications, that's not
> a problem, because there will be WAL records for those pages that need
> to be replayed, and we could recompute the CRC at the same time. But
> for hint bit changes, there's no persistent state that would tell us
> which hint bits were in the midst of being flipped when the system
> went down, so the only way to make sure all the CRCs are correct would
> be to rescan every page in the entire cluster and recompute every CRC.
>
> Is that right?

Yep.

Even if rescanning every page in the cluster was feasible from a
performance point-of-view, it would make the CRC checking a lot less
useful. It's not hard to imagine that when a hardware glitch happens
causing corruption, it also causes the system to crash. Recalculating
the CRCs after crash would mask the corruption.

--
Heikki Linnakangas
EnterpriseDB http://www.enterprisedb.com