Re: prevent immature WAL streaming

On 2021/09/01 0:53, Andres Freund wrote:
> I was thinking that on a normal WAL write we'd do nothing. Instead we would
> have dedicated code at the end of recovery that, if the WAL ends in a partial
> record, changes the page following the "valid" portion of the WAL to indicate
> that an incomplete record is to be skipped.

Agreed!

> Of course, we need to be careful to not weaken WAL validity checking too
> much. How about the following:
>
> If we're "aborting" a continued record, we set XLP_FIRST_IS_ABORTED_PARTIAL on
> the page at which we do so (i.e. the page after the valid end of the WAL).

When do you expect that XLP_FIRST_IS_ABORTED_PARTIAL is set? It's set
when recovery finds a a partially-flushed segment-spanning record?
But maybe we cannot do that (i.e., cannot overwrite the page) because
the page that the flag is set in might have already been archived. No?

> I think we can just read the WAL and see if it ends with a partial
> record. It'd add a bit of complication to the error checking in xlogreader,
> because we'd likely want to treat verification from page headers a bit
> different from verification due to record data. But that seems doable.

Yes.

> Does this make sense?

Yes, I think!

Regards,

--
Fujii Masao
Advanced Computing Technology Center
Research and Development Headquarters
NTT DATA CORPORATION