Re: Don't keep closed WAL segment in page cache after replay

From: Anthonin Bonnefoy <anthonin(dot)bonnefoy(at)datadoghq(dot)com>
To: Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com>
Cc: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Don't keep closed WAL segment in page cache after replay
Date: 2025-07-03 12:57:25
Message-ID: CAO6_Xqpx1ey8DPRub7fq4_-rtL=gygoSDHv4iLw5S3gG7=k1dw@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Thanks for the comments!

On Wed, Jul 2, 2025 at 7:12 PM Fujii Masao <masao(dot)fujii(at)oss(dot)nttdata(dot)com> wrote:
> WAL files that have already been replayed can still be read again
> for WAL archiving (if archive_mode = always) or for replication
> (if the standby is acting as a streaming replication sender or
> a logical replication publisher). No?
>
> Also, the WAL summarizer might read those WAL files as well.

True, I've forgotten we could do this on standbys. For archive_mode
and WAL summarizer, the check can be done with "XLogArchiveMode !=
ARCHIVE_MODE_ALWAYS" and "!summarize_wal".

For the replication, it looks a bit trickier. Checking if there's an
active walsender process seems like a good approach but I haven't
found an existing way to do this. Checking WalSndCtl->walsnds for used
slots isn't great as this should stay in walsender_private.h. A better
way would be to check how many elements there are in
ProcGlobal->walsenderFreeProcs. If there's max_wal_sender elements in
the list, then it means there's no active walsender processes. There's
already HaveNFreeProcs that could provide this information, though
it's currently only doing this for the freeProcs list. I've modified
HaveNFreeProcs to take a proc_free_list type as argument so it can be
used to get the number of free slots in walsenderFreeProcs.

One possible impact of this approach is when the cascading replication
stream starts (either first time or after a disconnect), the WAL files
will need to be read from disk instead of being already in the page
cache. Though I think that's a better default behaviour as I would
expect that most replicas don't have cascading replication and
removing closed WAL segments would benefit most replicas.

Regards,
Anthonin Bonnefoy

Attachment Content-Type Size
v02-0001-Don-t-keep-closed-WAL-segments-in-page-cache-aft.patch application/octet-stream 5.4 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2025-07-03 13:21:33 Re: Explicitly enable meson features in CI
Previous Message Amit Kapila 2025-07-03 10:51:31 Re: Conflict detection for update_deleted in logical replication