Re: Proposed WAL changes

"Mikheev, Vadim" <vmikheev(at)SECTORBASE(dot)COM> writes:
> If fsync may return before data *actually* flushed then you may have
> unlogged data page changes which breakes WAL rule and means corrupted
> (inconsistent) database without ANY ABILITY TO RECOVERY TO CONSISTENT
> STATE. Now please explain me how saving positions of two checkpoints
> (what Tom is doing) can help here?

If the WAL log is broken then there is (AFAICT) no way to *guarantee*
recovery to a consistent state. There might be changes in heap or index
pages that you cannot find a trace of in the readable part of the log,
and hence don't know you need to fix.

However, I think that Vadim is focusing too narrowly on what he can
guarantee, and neglecting the problem of providing reasonable behavior
when things are too broken to meet the guarantee. In particular,
complete refusal to start up when the WAL log is corrupted is not
reasonable nor acceptable. We have to be able to recover from that
and provide access to as-close-to-accurate data as we can manage.
I believe that backing up to the prior checkpoint and replaying as much
of the log as we can read is a useful fallback capability if the most
recent checkpoint record is unreadable.

In any case, "guarantees" that are based on the assumption that WAL
writes hit the disk before data writes do are going to fail anyway
if we are using disk drives that reorder writes. This is the real
reason why we can't think only about guarantees, but also about how
we will respond to "guaranteed impossible" failures.

If you can improve on this recovery idea, by all means do so. But
don't tell me that we don't need to worry about it, because we do.

regards, tom lane