Re: [HACKERS] [PATCH]make pg_rewind to not copy useless WAL files

Greetings,

* chenhj (chjischj(at)163(dot)com) wrote:
>
> At 2018-01-23 09:56:48, "Stephen Frost" <sfrost(at)snowman(dot)net> wrote:
> >I've only read through the thread to try and understand what's going on
> >and the first thing that comes to mind is that you're changing
> >pg_rewind to not remove the WAL from before the divergence (split)
> >point, but I'm not sure why. As noted, that WAL isn't needed for
> >anything (it's from before the split, after all), so why keep it? Is
> >there something in this optimization that depends on the old WAL being
> >there and, if so, what and why?
>
> After run pg_rewind, the first startup of postgres will do crash recovery.
> And crash recovery will begin from the previous redo point preceding the divergence.
> So, the WAL after the redo point and before the divergence is needed.

Right.

> Of course, the WAL before the redo point is not needed, but in my point of view,
> recycling unwanted WAL does not have to be done by pg_rewind.

That's what pg_rewind has been doing though, isn't it? And it's not
like that WAL is useful for anything, is it? That's also how
pg_basebackup works.

> >That's also different from how pg_basebackup works, which I don't think
> >is good (seems like pg_rewind should operate in a pretty similar manner
> >to pg_basebackup).
>
> Thanks for your comments!
> I also considered copy WAL just like how pg_basebackup does,but a implement similar to pg_basebackup's manner may be not so simple.

Using the replication protocol to fetch WAL would be a good thing to do
(actually, making pg_rewind entirely work through a connection to the
current primary would be great) but that's independent of what I'm
asking for here. Here I'm just suggesting that we not change what
pg_rewind is doing today when it comes to the existing WAL on the
old-primary.

> And the WAL which contains the previous redo point preceding the divergence may be only exists in target server and had been recycled in source. That's different between pg_rewind and pg_basebackup.

Hm, pg_rewind was removing that and expecting it to be on the new
primary? If that's the case then I could see an argument for keeping
WAL that's from the divergence point onward, but I still don't think we
should have pg_rewind just leave all of the prior WAL in place.

Thanks!

Stephen