Re: Race between KeepFileRestoredFromArchive() and restartpoint

From: David Steele <david(at)pgmasters(dot)net>
To: Noah Misch <noah(at)leadboat(dot)com>, pgsql-hackers(at)postgresql(dot)org, Kyotaro Horiguchi <horikyota(dot)ntt(at)gmail(dot)com>
Subject: Re: Race between KeepFileRestoredFromArchive() and restartpoint
Date: 2022-07-26 11:21:29
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi Noah,

On 6/19/21 16:39, Noah Misch wrote:
> On Tue, Feb 02, 2021 at 07:14:16AM -0800, Noah Misch wrote:
>> Recycling and preallocation are wasteful during archive recovery, because
>> KeepFileRestoredFromArchive() unlinks every entry in its path. I propose to
>> fix the race by adding an XLogCtl flag indicating which regime currently owns
>> the right to add long-term pg_wal directory entries. In the archive recovery
>> regime, the checkpointer will not preallocate and will unlink old segments
>> instead of recycling them (like wal_recycle=off). XLogFileInit() will fail.
> Here's the implementation. Patches 1-4 suffice to stop the user-visible
> ERROR. Patch 5 avoids a spurious LOG-level message and wasted filesystem
> writes, and it provides some future-proofing.
> I was tempted to (but did not) just remove preallocation. Creating one file
> per checkpoint seems tiny relative to the max_wal_size=1GB default, so I
> expect it's hard to isolate any benefit. Under the old checkpoint_segments=3
> default, a preallocated segment covered a respectable third of the next
> checkpoint. Before commit 63653f7 (2002), preallocation created more files.

This also seems like it would fix the link issues we are seeing in [1].

I wonder if that would make it worth a back patch?


In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2022-07-26 11:28:21 Re: make -C libpq check fails obscurely if tap tests are disabled
Previous Message Simon Riggs 2022-07-26 10:50:09 Re: Max compact as an FSM strategy