Re: Changing WAL Header to reduce contention during ReserveXLogInsertLocation()

On 27 March 2018 at 14:58, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> The point of the xl_prev links is, essentially, to allow verification
> that we are reading a coherent series of WAL records, ie that record B
> which appears to follow record A actually is supposed to follow it.
> This throws away that cross-check just as effectively as the other patch
> did, leaving us reliant solely on the per-record CRCs. CRCs are good,
> but they do have a false-positive rate.

You aren't reliant solely on the CRC.

If you have a false-positive CRC then xl_curr allows you to validate
the record, just as you would with xl_prev. The xl_curr value points
to itself using just as many bits as the xl_prev field, so the
probability of false validation is the same as now.

xl_prev does allow you to verify that the WAL records form an unbroken
chain. The question is whether that has value because it certainly has
a very clear cost in terms of write performance. Is there some danger
that we would skip records? Where does that risk come from? If someone
*wanted* to skip records for whatever reason, they could achieve that
with both xl_prev or xl_curr, so there is no difference there.

> An example of a case where xl_prev makes one feel a lot more comfortable
> is the WAL-segment-switch option. The fact that the first record of the
> next WAL file links back to the XLOG_SWITCH record is evidence that
> ignoring multiple megabytes of WAL was indeed the intended thing to do.
> An xl_curr field cannot provide any evidence as to whether WAL records
> were missed.

That's a good point, though very frequently unused since streaming replication.

We can handle XLOG_SWITCH as a special case. Perhaps we can make each
file header point back to the previous WAL record.

>> 2. Does the new logic in pg_rewind to search backward for a checkpoint
>> work reliably, and will it be slow?
>
> If you have to search backwards, this breaks it. Full stop.

You don't have to search backwards. We only need to locate the last
checkpoint record.

--
Simon Riggs http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services