Skip site navigation (1) Skip section navigation (2)

Re: Block-level CRC checks

From: Sam Mason <sam(at)samason(dot)me(dot)uk>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Block-level CRC checks
Date: 2008-10-01 19:03:05
Message-ID: 20081001190305.GQ7271@frubble.xen.chris-lamb.co.uk (view raw or flat)
Thread:
Lists: pgsql-hackers
On Wed, Oct 01, 2008 at 11:57:31AM -0400, Alvaro Herrera wrote:
> Tom Lane escribió:
> > "Jonah H. Harris" <jonah(dot)harris(at)gmail(dot)com> writes:
> 
> > > I probably wouldn't compare checksumming *every* WAL record to a
> > > single block-level checksum.
> > 
> > No, not at all.  Block-level checksums would be an order of magnitude
> > more expensive: they're on bigger chunks of data and they'd be done more
> > often.
> 
> More often?  My intention is that they are checked when the buffer is
> read in, and calculated/stored when the buffer is written out.
> In-memory changers of the block do not check nor recalculate the sum.

I know you said detecting memory errors wasn't being attempted, but
bad memory accounts for a reasonable number of reports of database
corruption on -general so I was wondering if moving the checks around
could catch some of these.

How about updating the block's checksum immediately whenever a tuple is
modified within it?  Checksums would be checked whenever data is read in
and just before it's written out.  Checksum failure on write would cause
PG to abort noisily with complaints about bad memory?

If some simple checksum could be found (probably a parity check) that
would allow partial updates PG wouldn't need to scan too much data when
regenerating it.  It would also cause the checksum to stay bad if data
goes bad in memory between reading from disk and its eventual write out,
a block remaining in cache for a large amount of time makes this case
appear possible.


  Sam

In response to

Responses

pgsql-hackers by date

Next:From: Kevin GrittnerDate: 2008-10-01 19:17:03
Subject: Re: Block-level CRC checks
Previous:From: Josh BerkusDate: 2008-10-01 17:47:07
Subject: September CommitFest Closed

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group