RE: Speed up the removal of WAL files

Thanks, it seems to require a bit more consideration about RemoveOldXLogFiles(). Let me continue this next month.

Regards
Takayuki Tsunakawa

> -----Original Message-----
> From: Michael Paquier [mailto:michael(dot)paquier(at)gmail(dot)com]
> Sent: Saturday, November 18, 2017 10:37 PM
> To: Fujii Masao
> Cc: Tsunakawa, Takayuki/綱川 貴之; Kyotaro HORIGUCHI;
> pgsql-hackers(at)postgresql(dot)org
> Subject: Re: Speed up the removal of WAL files
>
> On Sat, Nov 18, 2017 at 2:57 AM, Fujii Masao <masao(dot)fujii(at)gmail(dot)com> wrote:
> > On Fri, Nov 17, 2017 at 5:20 PM, Tsunakawa, Takayuki
> > <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:
> >> From: Kyotaro HORIGUCHI [mailto:horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp]
> >>> The orinal code recycles some of the to-be-removed files, but the
> >>> patch removes all the victims. This may impact on performance.
> >>
> >> Yes, I noticed it after submitting the patch and was wondering what to
> do. Thinking simply, I think it would be just enough to replace
> durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and
> sync the pg_wal directory once in RemoveNonParentXlogFiles() and
> RemoveOldXlogFiles(). This will benefit the failover time when fast
> promotion is not performed. What do you think?
>
> Note that durable_rename() also flushes the origin file to make sure that
> it does not show up again after a crash.
>
> > It seems not good idea to just replace durable_rename() with rename()
> > in RemoveOldXlogFiles()->RemoveXlogFiles()->InstallXLogFileSegment().
> > Because that change seems to be able to cause the following problem.
>
> If archiving is enabled, RemoveOldXlogFiles() would create a .ready flag
> for all segments that have reappeared. Oops, it archived again.