Re: index prefetching

On Wed, Jul 16, 2025 at 6:18 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
> There's no problem today - the indexams never use the tids to look up blocks
> themselves. They're always passed to the tableam to do so (via
> table_index_fetch_tuple() etc). I.e. the translation from TIDs to specific
> blocks & buffers happens entirely inside the tableam, therefore the tableam
> can choose to not use a 1:1 mapping or even to not use any buffers at all.

Of course. Somehow, I missed that obvious point. That is the bare
minimum for a new interface such as this.

> ISTM the right answer would be to allow the tableam to get the batches,
> without indexam feeding the read stream. That, perhaps not so coincidentally,
> is also what's needed for batching heap page locking and and HOT search.

I agree.

> I think this means that it has to be the tableam that creates the read stream
> and that does the work that's currently done in index_scan_stream_read_next(),
> i.e. the translation from TID to whatever resources are required by the
> tableam. Which presumably would include the tableam calling
> index_batch_getnext().

It probably makes sense to put that off for (let's say) a couple more
months. Just so we can get what we have now in better shape. The
"complex" patch only very recently started to pass all my tests (my
custom nbtree test suite used for my work in 17 and 18).

I still need buy-in from Tomas on the "complex" approach. We chatted
briefly on IM, and he seems more optimistic about it than I thought
(in my on-list remarks from earlier). It is definitely his patch, and I don't
want to speak for him.

--
Peter Geoghegan