Re: Use durable_unlink for .ready and .done files for WAL segment removal

Greetings,

* Michael Paquier (michael(at)paquier(dot)xyz) wrote:
> On Fri, Sep 28, 2018 at 02:36:19PM -0400, Stephen Frost wrote:
> > Is there an issue with making the archiver able to understand that
> > situation instead of being confused by it..? Seems like that'd probably
> > be a good thing to do regardless of this, but that would then remove the
> > need for this kind of change..
>
> I thought about that a bit, and there is as well a lot which can be done
> within the archive_command itself regarding that, so I am not sure that
> there is the argument to make pgarch.c more complicated than it should.
> Now it is true that for most users having a .ready file but no segment
> would most likely lead in a failure. I suspect that a large user base
> is still just using plain cp in archive_command, which would cause the
> archiver to be stuck. So we could actually just tweak pgarch_readyXlog
> to check if the segment fetched actually exists (see bottom of the
> so-said function). If it doesn't, then the archiver removes the .ready
> file and retries fetching a new segment.

Yes, checking if the WAL file exists before calling archive_command on
it is what I was thinking we'd do here, and if it doesn't, then just
remove the .ready file.

An alternative would be to go through the .ready files on crash-recovery
and remove any .ready files that don't have corresponding WAL files, or
if we felt that it was necessary, we could do that on every restart but
do we really think we'd need to do that..?

Thanks!

Stephen