Re: index prefetching

Hi,

On 2025-08-28 19:57:17 -0400, Peter Geoghegan wrote:
> On Thu, Aug 28, 2025 at 7:52 PM Tomas Vondra <tomas(at)vondra(dot)me> wrote:
> > Use this branch:
> >
> > https://github.com/tvondra/postgres/commits/index-prefetch-master/
> >
> > and then Thomas' patch that increases the prefetch distance:
> >
> >
> > https://www.postgresql.org/message-id/CA%2BhUKGL2PhFyDoqrHefqasOnaXhSg48t1phs3VM8BAdrZqKZkw%40mail.gmail.com
> >
> > (IIRC there's a trivial conflict in read_stream_reset.).
>
> I found it quite hard to apply Thomas' patch. There's actually 3
> patches, with 2 earlier patches needed for earlier in the thread. And,
> there were significant merge conflicts to work around.

Same. Tomas, could you share what you applied?

> I'm not sure that Thomas'/your patch to ameliorate the problem on the
> read stream side is essential here. Perhaps Andres can just take a
> look at the test case + feature branch, without the extra patches.
> That way he'll be able to see whatever the immediate problem is, which
> might be all we need.

It seems caused to a significant degree by waiting at low queue depths. If I
comment out the stream->distance-- in read_stream_start_pending_read() the
regression is reduced greatly.

As far as I can tell, after that the process is CPU bound, i.e. IO waits don't
play a role.

I see a variety for increased CPU usage:

1) The private ref count infrastructure in bufmgr.c gets a bit slower once
more buffers are pinned

2) signalling overhead to the worker - I think we are resetting the latch too
eagerly, leading to unnecessarily many signals being sent to the IO worker.

3) same issue with the resowner tracking

But there's some additional difference in performance I don't yet
understand...

Greetings,

Andres Freund