Re: Block-level CRC checks

On Tue, 2009-12-01 at 15:30 +0200, Heikki Linnakangas wrote:
> Bruce Momjian wrote:
> > What might be interesting is to report CRC mismatches if the database
> > was shut down cleanly previously; I think in those cases we shouldn't
> > have torn pages.
>
> Unfortunately that's not true. You can crash, leading to a torn page,
> and then start up the database and shut it down cleanly. The torn page
> is still there, even though the last shutdown was a clean one.

There seems to be two ways forwards: journalling or fsck.

We can either

* WAL-log all changes to a page (journalling) (8-byte overhead)

* After a crash disable CRC checks until a full database scan has either
re-checked CRC or found CRC mismatch, report it in the LOG and then
reset the CRC. (fsck) (8-byte overhead)

Both of which can be optimised in various ways.

Also, we might

* Put all hint bits in the block header to allow them to be excluded
more easily from CRC checking. If we used 3 more bits from
ItemIdData.lp_len (limiting tuple length to 4096) then we could store
some hints in the item pointer. HEAP_XMIN_INVALID can be stored as
LP_DEAD, since that will happen very quickly anyway.

--
Simon Riggs www.2ndQuadrant.com