On Tue, 2009-12-01 at 13:05 -0500, Bruce Momjian wrote:
> Simon Riggs wrote:
> > Also, we might
> > 
> > * Put all hint bits in the block header to allow them to be excluded
> > more easily from CRC checking. If we used 3 more bits from
> > ItemIdData.lp_len (limiting tuple length to 4096) then we could store
> > some hints in the item pointer. HEAP_XMIN_INVALID can be stored as
> > LP_DEAD, since that will happen very quickly anyway. 
> 
> OK, here is another idea, maybe crazy:

When there's nothing else left, crazy wins.

> When we read in a page that has an invalid CRC, we check the page to see
> which hint bits are _not_ set, and we try setting them to see if can get
> a matching CRC.  If there no missing hint bits and the CRC doesn't
> match, we know the page is corrupt.  If two hint bits are missing, we
> can try setting one and both of them and see if can get a matching CRC. 
> If we can, the page is OK, if not, it is corrupt.
> 
> Now if 32 hint bits are missing, but could be based on transaction
> status, then we would need 2^32 possible hint bit combinations, so we
> can't do the test and we just assume the page is valid.
> 
> I have no idea what percentage of corruption this would detect, but it
> might have minimal overhead because the overhead only happens when we
> detect a non-matching CRC due to a crash of some sort.

Perhaps we could store a sector-based parity bit each 512 bytes in the
block. If there are an even number of hint bits set, if odd we unset the
parity bit. So whenever we set a hint bit we flip the parity bit for
that sector. That way we could detect which sectors are potentially
missing in an effort to minimize the number of combinations we need to
test. That would require only 16 bits for an 8192 byte block; we store
it next to the CRC, so we know that was never altered separately. So
total 6 byte overhead.

-- 
 Simon Riggs           www.2ndQuadrant.com

In response to

• Re: Block-level CRC checks at 2009-12-01 18:05:47 from Bruce Momjian

Responses

• Re: Block-level CRC checks at 2009-12-01 18:58:35 from Tom Lane

pgsql-hackers by date

Next:	From: Tom Lane	Date: 2009-12-01 18:25:22
	Subject: Re: Application name patch - v4
Previous:	From: Robert Haas	Date: 2009-12-01 18:20:08
	Subject: Re: Block-level CRC checks