Re: BUG: ReadStream look-ahead exhausts local buffers when effective_io_concurrency>=64

On Fri, Jul 24, 2026 at 1:40 AM Xuneng Zhou <xunengzhou(at)gmail(dot)com> wrote:
>
> On Thu, Jul 23, 2026 at 11:08 PM Melanie Plageman
> <melanieplageman(at)gmail(dot)com> wrote:
> >
>
> > I wonder if in 20, we should move toward Andres' suggested formula
> > Max(1, (num_temp_buffers − NLocalPinnedBuffers) / 4)
> > instead of the existing
> > num_temp_buffers / 4 − NLocalPinnedBuffers
> >
> > Which would mean we got a quarter of available buffers instead of a
> > hard cap of a quarter of total buffers.
> > This wouldn't have been any better at solving the issue in this
> > thread, but I'm wondering if it is just a better formula in general
> > going forward (not for backpatching).
>
> I am not sure about that. In the existing formula, we set a hard-cap
> for the read streams at backend level. For the suggested one, the cap
> is removed so multiple successive streams could push the water level
> of pins toward the ceiling of temp buffers. I am unsure of the
> consequences of this. Adjusting the cap limit alone seems not resolve
> the competing nature between read stream and other needs. Reclaiming
> some pinned buffers might help, but it is a more complex mechanism
> which might not be a good fit in the bigger picture.

I think the right approach is to drive out a change like this with
specific use cases or after seeing bottlenecks/problems in workloads.
Since we're not encountering those now, I think investigating how to
change this falls far down the priority list.

- Melanie