Re: index prefetching

On Mon, Apr 6, 2026 at 9:09 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
> Quite a few people have tried very hard over the last months to get this, and
> its dependencies, into a committable shape. The patchset is much better than
> it was. The performance is better. Regressions are drastically reduced. The
> performance gains are often unbelievably large.
>
> I'd be happy to merge the patchset from the performance POV. I'd be happy to
> merge it from a high-level architectural perspective. It's just the pesky
> details that need more buffing out than our tired arms can do in the remaining
> time. I think Peter feels similar.

That's a good summary.

At a high level, the patch has no significant problems. It introduces
virtually no regressions, which required a great deal of effort from
all of us -- that exceeds what I once thought possible. There's a
general sense that the high level architecture is a big improvement,
without any notable flaws.

As you said, it came down to lots of small things. And timing. Many
last minute non-trivial changes to specific aspects of the patch
design seemed necessary. This all happened under enormous time
pressure. And I'm honestly just exhausted right now (I'm likely not
the only one). I just couldn't justify committing such an invasive
patch set under those circumstances.

Attached is v24 -- one last revision ahead of feature freeze, just to
post what I have, before taking some time away from this.

Summary of changes from v23:

* index_getnext_tid is renamed and moved into indexbatch.h, where it
becomes a utility function that has a similar contract to the one that
is used to call amgetbatch (and manage changes in the batch
direction). Now there's only one canonical call to
pgstat_count_index_tuples in heapam, and no accidental table index
fetch state resets when a scan that happens to be using amgettuple
ends.

* Miscellaneous small cleanups, including renaming some of the
functions in indexbatch.c.

* Slightly simplified tableam_util_batchscan_mark_pos logic.

Obviously this isn't everything that I'd hoped would appear in the
next revision. It's just what got done in the past 24 hours.

--
Peter Geoghegan