* Gregory Stark <stark(at)enterprisedb(dot)com> [081001 11:59]:
 
> If setting a hint bit cleared a flag on the buffer header then the
> checksumming process could set that flag, begin checksumming, and check that
> the flag is still set when he's finished.
> 
> Actually I suppose that wouldn't actually be good enough. He would have to do
> the i/o and check that the checksum was still valid after the i/o. If not then
> he would have to recalculate the checksum and repeat the i/o. That might make
> the idea a loser since I think the only way it wins is if you rarely actually
> get someone setting the hint bits during i/o anyways.

A doubled-write is essentially "free" with PostgreSQL because it's not
doing direct IO, rather relying on the OS page cache to be efficient.
So if you write block A and then call write on block A immediately (or,
realistically, after a redo of the checksum), the first write is almost
*never* going to take IO bandwidth to your spindles...

But the problem is if something crashes (or interrupts PG) between those
two writes, you've got a block of data into the pagecache (and possibly
to the disks) that PG will no longer read in, because the CRC/checksum
fails despite the actual content being valid...

So if you're going to be makeing PG refuse to read-in blocks with bad
CRC/csum, you need to guarnetee that nothing fiddles with the block
between the start of the CRC and the completion of the write().

One possibility would be to "double-buffer" the write... i.e. as you
calculate your CRC, you're doing it on a local copy of the block, which
you hand to the OS to write...  If you're touching the whole block of
memory to CRC it, it isn't *ridiculously* more expensive to copy the
memory somewhere else as you do it...

a.

-- 
Aidan Van Dyk                                             Create like a god,
aidan(at)highrise(dot)ca                                       command like a king,
http://www.highrise.ca/                                   work like a slave.

In response to

• Re: Block-level CRC checks at 2008-10-01 15:57:50 from Gregory Stark

Responses

• Re: Block-level CRC checks at 2008-10-01 17:00:36 from Mark Mielke
• Re: Block-level CRC checks at 2008-10-01 17:25:52 from Gregory Stark
• Re: Block-level CRC checks at 2008-10-01 20:16:13 from Tom Lane

pgsql-hackers by date

Next:	From: Paul Schlie	Date: 2008-10-01 16:27:35
	Subject: Re: Block-level CRC checks
Previous:	From: Andreas Kretschmer	Date: 2008-10-01 16:16:03
	Subject: Re: Transactions within a function body