Re: archive status ".ready" files may be created too early

On 12/17/19, 2:26 AM, "Kyotaro Horiguchi" <horikyota(dot)ntt(at)gmail(dot)com> wrote:
> But I think I found a corner case where the patch doesn't work. As I
> mentioned in another message, if WAL buffer was full,
> AdvanceXLInsertBuffer calls XLogWrite to write out the victim buffer
> regardless whether the last record in the page was the first half of a
> continuation record. XLogWrite can mark the segment as .ready even
> with the patch.
>
> Is that correct? And do you think the corner case is worth amending?

I certainly think it is worth trying to prevent potential WAL archive
corruption in known corner cases. Your comment highlights a potential
shortcoming of my patch. AFAICT there is no guarantee that
XLogWrite() is called with a complete WAL record. Even if that
assumption is true at the moment, it might not hold up over time.

> If so, we could amend also that case by marking the last segment as
> .ready when XLogWrite writes the first bytes of the next segment. (As
> the further corner-case, it still doesn't work if a contination record
> spans over trhee or more segments.. But I don't (or want not to) think
> we don't need to consider that case..)

I'm working on a new version of the patch that will actually look at
the WAL page metadata to determine when it is safe to mark a segment
as ready for archival. It seems relatively easy to figure out whether
a page is the last one for the current WAL record.

Nathan