Re: archive status ".ready" files may be created too early

At Wed, 16 Dec 2020 11:01:20 +0900 (JST), Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com> wrote in
> - Record the beginning LSN of the first cross-seg record and the end
> LSN of the last cross-seg recrod in a consecutive segments bonded by
> cross-seg recrods. Spcifically X and Y below.
>
> X Z Y
> [recA] [recB] [recC]
> [seg A] [seg B] [seg C] [seg D] [seg E]
> (1) (2.2) (2.2) (2.1) (2.1) (1)
>
> 1. If we wrote upto before X or beyond Y at a segment boundary, notify
> the finished segment immediately.
>
> 1.1. If we have written beyond Y, clear the recorded region.
>
> 2. Otherwise we don't notify the segment immediately:
>
> 2.1. If write request was up to exactly the current segment boundary
> and we know the end LSN of the record there (that is, it is recC
> above), extend the request to the end LSN. Then notify the segment
> after the record is written to the end.
>
> 2.2. Otherwise (that is recA or recB), we don't know whether the
> last record of the last segment is ends just at the segment boundary
> (Z) or a record spans between segments (recB). Anyway even if there
> is such a record there, we don't know where it ends. As the result
> what we can do there is only to refrain from notifying. It doesn't
> matter so much since we have already inserted recC so we will soon
> reach recC and will notify up to seg C.
>
> There might be a case where we insert up to Y before writing up to Z,
> the segment-region X-Y contains non-connected segment boundary in that
> case. It is processed as if it is a connected segment
> boundary. However, like 2.2 above, It doesn't matter since we write up
> to Y soon.

I noticed that we can cause the continuation record flushed
immedately. So in the attached,

1. If there's no remembered cross-segment boundary or we're out of the
region X-Y, notify the finished segment immediately.

2. Otherwise we don't notify the segment immedately

2.1. If we are finishing the last semgment known to continue to the
next segment, extend write request to the end of the recrod *and*
force to write then flush up to there.

2.2. (the same to the above)

3. In the case of 2.1, we can flush the previous segment immediately
so do that.

X. When we notify a segment, clear the rememberd region if we have got
out of the region.

The attached is changed in the following points:

- Fixed some bugs that I confusedly refer to write-lsn instead of flush-lsn.

- Changed to urge flushing up to the end of a continuation record, not
only waiting for the recrod to be written.

- More agressively clear the remembered region.

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center