Skip site navigation (1) Skip section navigation (2)

Re: Block-level CRC checks

From: pgsql(at)mohawksoft(dot)com
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: pgsql(at)mohawksoft(dot)com, "Hannu Krosing" <hannu(at)2ndquadrant(dot)com>, "Decibel!" <decibel(at)decibel(dot)org>, "Alvaro Herrera" <alvherre(at)commandprompt(dot)com>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Block-level CRC checks
Date: 2008-10-01 15:37:35
Message-ID: 60703.71.232.149.185.1222875455.squirrel@mail.mohawksoft.com (view raw or flat)
Thread:
Lists: pgsql-hackers
> pgsql(at)mohawksoft(dot)com writes:
>>> No, it's all about time penalties and loss of concurrency.
>
>> I don't think that the amount of time it would take to calculate and
>> test
>> the sum is even important. It may be in older CPUs, but these days CPUs
>> are so fast in RAM and a block is very small. On x86 systems, depending
>> on
>> page alignment, we are talking about two or three pages that will be "in
>> memory" (They were used to read the block from disk or previously
>> accessed).
>
> Your optimism is showing ;-).  XLogInsert routinely shows up as a major
> CPU hog in any update-intensive test, and AFAICT that's mostly from the
> CRC calculation for WAL records.
>
> We could possibly use something cheaper than a real CRC, though.  A
> word-wide XOR (ie, effectively a parity calculation) would be sufficient
> to detect most problems.

That was something that I mentioned in my first response. if the *only*
purpose of the check is to generate a "pass" or "fail" status, and not
something to be used to find where in the block it is corrupted or attempt
to regenerate the data, then we could certainly optimize the check
algorithm. A simple checksum may be good enough.

In response to

pgsql-hackers by date

Next:From: Reg Me PleaseDate: 2008-10-01 15:54:01
Subject: Transactions within a function body
Previous:From: Tom LaneDate: 2008-10-01 15:36:44
Subject: Re: Block-level CRC checks

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group