Re: Block-level CRC checks

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Simon Riggs <simon(at)2ndQuadrant(dot)com>, Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>, Aidan Van Dyk <aidan(at)highrise(dot)ca>, Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Block-level CRC checks
Date: 2009-12-01 21:49:06
Message-ID: 200912012149.nB1Ln6V12367@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Bruce Momjian <bruce(at)momjian(dot)us> writes:
> > OK, here is another idea, maybe crazy:
>
> > When we read in a page that has an invalid CRC, we check the page to see
> > which hint bits are _not_ set, and we try setting them to see if can get
> > a matching CRC. If there no missing hint bits and the CRC doesn't
> > match, we know the page is corrupt. If two hint bits are missing, we
> > can try setting one and both of them and see if can get a matching CRC.
> > If we can, the page is OK, if not, it is corrupt.
>
> > Now if 32 hint bits are missing, but could be based on transaction
> > status, then we would need 2^32 possible hint bit combinations, so we
> > can't do the test and we just assume the page is valid.
>
> A typical page is going to have something like 100 tuples, so
> potentially 2^400 combinations to try. I don't see this being
> realistic from that standpoint. What's much worse is that to even
> find the potentially missing hint bits, you need to make very strong
> assumptions about the validity of the rest of the page.
>
> The suggestions that were made upthread about moving the hint bits
> could resolve the second objection, but once you do that you might
> as well just exclude them from the CRC and eliminate the guessing.

OK, crazy idea #3. What if we had a per-page counter of the number of
hint bits set --- that way, we would only consider a CRC check failure
to be corruption if the count matched the hint bit count on the page.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2009-12-01 21:52:26 Re: A thought about regex versus multibyte character sets
Previous Message Alvaro Herrera 2009-12-01 21:46:11 Re: A thought about regex versus multibyte character sets