Re: Checkpoint cost, looks like it is WAL/CRC

Bruce Momjian wrote:
>
> I don't think our problem is partial writes of WAL, which we already
> check, but heap/index page writes, which we currently do not check for
> partial writes.

Hmm...I've read through the thread again and thought about the problem
further, and now think I understand what you're dealing with.

Ultimately, the problem is that you're storing diffs in the WAL, so
you have to be able to guarantee that every data/index page has been
completely written, right?

There's no way to detect a torn page without some sort of marker in
each disk-indivisible segment of the page, unless you're willing to
checksum the entire page. With that in mind, the method Microsoft
uses for SQL Server is probably about as simple as it gets. In our
case, I suppose we may as well allocate one byte per 256-bytes segment
for the torn page marker. Just increment the marker value each time
you write the page (you'll have to read it from the page prior to
incrementing it, of course).

Other than that, torn page detection is really just a special case of
page corruption detection.

--
Kevin Brown kevin(at)sysexperts(dot)com