Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64

On Thu, Jul 23, 2026 at 5:17 PM Feike Steenbergen
<feikesteenbergen(at)gmail(dot)com> wrote:
>
>
>
> On Fri, 3 Jul 2026 at 05:54, Xuneng Zhou <xunengzhou(at)gmail(dot)com> wrote:
>
> > > On Wed, 6 May 2026 at 14:24, Feike Steenbergen <feikesteenbergen(at)gmail(dot)com> wrote:
> > > > For now, the problem disappears again when switching to io_method='worker'
> > >
> > > I spoke too soon it seems. It happens less frequently, but we do still have the issue even with io_method='worker'.
> > > Reverting to io_method='sync' as I would expect that will solve the issue.
> >
> > Interestingly, I wonder why this variance would occur in production.
> > This is the more interesting part of this thread. I did some
> > investigation & experiment, pg 19 passed the reproducer even if
> > da6874635db is removed. A series of improvements made in April may
> > contribute to this. Will return soon to share the findings..
>
> Indeed, we see the problem occur at *all* settings of io_method. As it is slightly unpredictable,
> a retry of a transaction may succeed afterwards, so our production application is mostly
> fine. We did observe these errors right after upgrading to pg19, then we didn't observe the error
> for a full 3 weeks, but now we observe the error dozens of times every hour.
>
> Our application has a bit of a cadence though, it only does its work 5/6 hours, and the error
> is only showing when the application is actually doing work.

That seems strange. Can you share some failure logs and describe the
workload a bit more? There are possible workloads that keep squeezing
out the temp buffers. It would also be helpful to do some quick
diagnoses like:

1) ALTER SYSTEM SET backtrace_functions = 'GetLocalVictimBuffer';
SELECT pg_reload_conf();

2) SELECT version();

3) Settings: temp_buffers, effective_io_concurrency, io_combine_limit

> > 2) Three engines, two pools
> >
> > To be continued...
>
> On our end, we're trying to reduce connections (batch more, queue work outside the db more),
> however, it would be great if this problem can be fully identified and resolved in pg
> itself.

If this same issue/failure persists, I'm afraid this move would not
mitigate it or could be counterproductive since the temp-pressure per
backend is bigger. It might be more helpful to up the temp_buffers &
down the effective_io_concurrency if the arithmetic applies here.

--
Regards,
Xuneng Zhou
HighGo Software Co., Ltd.