Re: pg_rewind : feature to rewind promoted standby is broken!

On Tue, Mar 12, 2019 at 01:23:51PM +0530, Mithun Cy wrote:
> I think pg_rewind's feature to rewind the promoted standby as a new
> standby is broken in 11

Confirmed, it is.

> Also I have tested same in version 10 it works fine there.
>
> Did below commit has broken this feature? (Thanks to kuntal for
> identifying same)
> commit 266b6acb312fc440c1c1a2036aa9da94916beac6
> Author: Fujii Masao <fujii(at)postgresql(dot)org>
> Date: Thu Mar 29 04:56:52 2018 +0900
> Make pg_rewind skip files and directories that are removed during server start.

And you are pointing out to the correct commit. The issue is that
process_target_file() has added a call to check_file_excluded(), and
this skips all the folders which it thinks can be skipped. One
problem though is that we also filter out pg_internal.init, which is
present in each database folder, and remains in the target directory
marked for deletion. Then, when the deletion happens, the failure
happens as the directory is not fully empty.

We could consider using rmtree() instead, but it is a nice sanity
check to make sure that all the entries in a path have been marked for
deletion. Just removing the filter check on the target is fine I
think, as we only should have the filters anyway to avoid copying
unnecessary files from the source. Attached is a patch. What do you
think?

(check_file_excluded() could be simplified further but it's also nice
to keep some mirroring in this API if we finish by using it for target
files at some point.)
--
Michael