RE: Speed up the removal of WAL files

From: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>
To: 'Kyotaro HORIGUCHI' <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
Cc: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: Speed up the removal of WAL files
Date: 2017-11-17 08:20:36
Message-ID: 0A3221C70F24FB45833433255569204D1F81B241@G01JPEXMBYT05
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

From: Kyotaro HORIGUCHI [mailto:horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp]
> The orinal code recycles some of the to-be-removed files, but the patch
> removes all the victims. This may impact on performance.

Yes, I noticed it after submitting the patch and was wondering what to do. Thinking simply, I think it would be just enough to replace durable_unlink/durable_rename in RemoveXLogFile() with unlink/rename, and sync the pg_wal directory once in RemoveNonParentXlogFiles() and RemoveOldXlogFiles(). This will benefit the failover time when fast promotion is not performed. What do you think?

BTW, RemoveNonParentXlogFiles() recycles only 10 WAL files and delete all other files. So the impact of modification is limited.

> Likewise the original code is using durable_unlink to actually remove a
> file so separating unlink and fsync might resurrect the problem that should
> have been fixed by
> 1b02be21f271db6bd3cd43abb23fa596fcb6bac3 (I'm not sure what it was but you
> are one of the reviwers of it). I suppose that you need to explain the reason
> why this change doesn't risk anything.

It's safe because the directory is finally synced. If the database server crashes before it deletes all WAL files, it performs the deletion again during the next recovery.

Regards
Takayuki Tsunakawa

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Marco Nenciarini 2017-11-17 09:09:53 Re: pspg - psql pager
Previous Message Kyotaro HORIGUCHI 2017-11-17 08:03:53 Re: Speed up the removal of WAL files