Re: Block-level CRC checks

On Fri, Oct 17, 2008 at 12:26:11PM -0300, Alvaro Herrera wrote:
> So this discussion died with no solution arising to the
> hint-bit-setting-invalidates-the-CRC problem.
>
> Apparently the only solution in sight is to WAL-log hint bits. Simon
> opines it would be horrible from a performance standpoint to WAL-log
> every hint bit set, and I think we all agree with that. So we need to
> find an alternative mechanism to WAL log hint bits.

There is another option I havn't seen mentioned anywhere yet: a single
bit change in a page has a predictable change on the CRC, dependant
only on the position of the bit. So in theory it would be possible for
the process changing the hint bit to update the CRC with a single XOR
operation. Working out what to XOR it with is the hard part.

Worst case you're talking about a BLOCK_SIZE*8*4 byte = 256K lookup
table but CRC has nice mathematical properties which could probably
get that down to a few KB.

Although, maybe locking of the hint bits would be a problem?

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> Please line up in a tree and maintain the heap invariant while
> boarding. Thank you for flying nlogn airlines.