Re: archive status ".ready" files may be created too early

On 8/4/21, 6:58 PM, "Kyotaro Horiguchi" <horikyota(dot)ntt(at)gmail(dot)com> wrote:
> Addition to that, while NotifySegmentsReadyForArchive() is notifying
> pending segments, other backends simultaneously reach there are
> blocked until the notification, incuding file creation, finishes. I
> don't think that's great. Couldn't we set lastNotifiedSegment before
> the loop? At the moment a backend decides to notify some segments,
> others no longer need to consider those segments. Even if the backend
> crashes meanwhile, as you mentionied below, it's safe since the
> unnotified segments are notifed after restart.

That seems reasonable to me. It looks like we rely on
RemoveOldXlogFiles() even today for when XLogArchiveNotify() fails. I
updated this in v4 of the patch.

In addition to this change, I also addressed your other feedback by
changing XLogSegNoIsInvalid() to XLogSegNoIsValid() and by moving
record boundary registration to the "if" block for cross-page records.

> Does it work that RegisterRecordBoundaryEntry omits registering of the
> bounary if it finds lastNotifiedSeg have gone too far?

Yeah, there's no reason to add a record boundary if we've already
notified the prior segment. For that to happen, another cross-segment
record would have to be flushed to disk and
NotifySegmentsReadyForArchive() would have to be called before
registering the boundary. With that being said, I don't expect an
extra map entry here and there to impact performance enough for us to
worry about it.

Nathan