Re: Use WALReadFromBuffers in more places

Hi,

On Sat, Sep 20, 2025 at 1:33 AM Jingtang Zhang <mrdrivingduck(at)gmail(dot)com> wrote:
>
> > Thanks for looking at this. Yes, the WAL writers can zero out flushed
> > buffers before WALReadFromBuffers gets to them. However,
> > WALReadFromBuffers was intentionally designed as an opportunistic
> > optimization - it's a "try this first, quickly" approach before
> > falling back to reading from WAL files. The no-locks design ensures it
> > never gets in the way of backends generating WAL, which is critical
> > for overall system performance.
>
> Yes, it is actually an interesting thing, beyond current topic. Since
> we are using buffered I/O, even though we cannot read from WAL buffer due
> to the opportunistic AdvanceXLInsertBuffer by WAL writer, later WALRead may
> still find the page inside OS page cache, with high probability, because the
> page has just been written out. So WALRead will be fast, too.

When there are logical replication (or logical decoding) clients that
are closely following the WAL insertion point (IOW, the consumers are
all well behaving and catching up with the publisher's WAL generation
rate), such walsenders reading from WAL buffers can still avoid
reading from WAL files (which can either be OS page cache reads or
disk reads) even before the WAL writer's opportunistic WAL buffer
initialization zeros out the WAL data. So it all depends on the
workload and how well the logical replication clients are catching up.

> But if we are moving forward to direct I/O some day in the future, the cost
> of WALReadFromBuffers and WALRead might be obvious.

Yes, that's correct. I did a quick experiment and will send more details soon.

> Maybe the opportunistic
> WAL buffer initialization could keep a small ratio of old pages inside WAL
> buffer so these pages can still be hit by WALReadFromBuffers.

This is an interesting idea. It could be discussed separately as
slowing down the WAL writer's opportunistic WAL buffer initialization
could have an impact on foreground write workloads.

> > I rebased and attached the v3 patch.
>
> The v3 patch LGTM.

Thanks for reviewing the patch. I will send the new patch with a fix
that I found in testing soon.

--
Bharath Rupireddy
Amazon Web Services: https://aws.amazon.com