Re: Perf: Specialize ExecIndexOnlyScan() like ExecSeqScan()

On Wed, Oct 14, 2026 at 12:14 PM Andres Freund <andres(at)anarazel(dot)de> wrote:
>
> Hi,
>
> On 2026-10-14 11:12:57 -0700, Haibo Yan wrote:
> > However, I wonder whether we should evaluate the broader tradeoffs
> > before introducing more hand-specialized executor variants. While
> > the optimization makes sense in isolation, I'm less certain about
> > its long-term benefits relative to the additional code complexity
> > and maintenance cost.
> >
> > In particular, it would be interesting to see how much of the
> > improvement survives with LTO/PGO enabled. Both can help with
> > inlining and code layout, potentially reducing the benefit of
> > manual specialization.
>
> I really don't see how it could obsolete the improvement in the patch (it
> could help at the same time obviously, but the patch should improve LTO and
> non-LTO). The compiler has no way of knowing that estate->es_epq_active,
> indexstate->ss.ps.qual, indexstate->ss.ps.ps_ProjInfo don't change during
> query execution, that's something only we know.
>
> It's possible that that some LTO implementations would end up inlining
> ExecSeqScan() and then convert the indirect function call to IndexOnlyNext()
> into a [potentially guarded] direct function call. But that's only a
> relatively small portion of the benefit of the specializations!
>
> There's is *some* support for speculative assumptions with PGO informed LTO,
> but to my knowledge that's for stuff like speculative devirtualization. I
> don't think it'd apply to anything like what we're dealing with here. But even
> if it did, it'd not help with specializations, as the problem is having to
> check all those branches in every IndexOnlyNext(), which in turn require a
> bunch of loads.
>
>
> > It would also be useful to measure the
> > incremental gain with JIT expression compilation enabled, even
> > though PostgreSQL's current JIT does not cover the scan loop itself.
>
> > I'm also concerned about the cumulative code-size cost. The
> > increase from roughly 0.6 KB to 3.5 KB may be acceptable for
> > IndexOnlyScan alone, but applying the same approach to IndexScan,
> > BitmapHeapScan, and other nodes could have broader implications
> > for instruction-cache locality and maintainability.
>
> Of course we shouldn't go overboard, but I'd expect this to *increase* icache
> locality in a broad range of use cases. Postgres' binary is way too big and
> our execution paths way too "wide" for relatively small increases in code size
> to harm meaningfully, when the same changes make the amount of code executed
> smaller in almost all cases. E.g. the EPQ handling code is never required in
> hot execution paths, and keeping that on separate cachelines will *increase*
> the actual icache effectiveness.
>
>
> > Finally, with ongoing work on batched execution and index access,
> > I think it would be worth understanding whether these optimizations
> > remain complementary or primarily target the same per-tuple
> > overhead. In particular, if batching amortizes executor dispatch
> > and function-call costs, how much additional benefit would
> > specialization provide?
>
> I think batching in ExecIndexOnlyScan() is unlikely to be very generally
> applicable anytime soon, because it's so often used in contexts like nested
> loops, where - unless you have a much more complicated join implementation -
> you will only want to fetch a single tuple on the inner side.
>
> I also think we're not that close to actually getting the batched execution
> stuff.
>
> And more importantly, even with the batched execution, and a lot of additional
> improvements, you'd *still* want specialization for having a qual, EPQ,
> projection, because checking them inside the loop over the tuples in the batch
> makes no sense. The only thing that'd actually be amortized is the indirect
> function call to *Next().
>
>
> Greetings,
>
> Andres Freund

Hi Andres, Matthias,

Thanks, Andres, for the detailed explanation. After looking into this
more carefully, I agree with your points.

In particular, I was conflating compiler inlining with specialization
based on executor-state invariants. LTO/PGO cannot generally establish
those invariants, and batching would not eliminate the need to avoid
per-tuple qual, projection, and EPQ checks. Your explanation of why
specialization can improve instruction-cache locality also makes sense.

I was still curious about the code-layout and frontend effects,
particularly under mixed workloads, so I ran some additional tests
rather than continuing to speculate.

On Linux x86-64 (i9-10900K, GCC 13.3), I compared unpatched master
against v2 with both default and 64-byte function alignment.

The main findings were:

Specialization consistently reduced retired instructions. For the
qual-only case, the reduction was about 5.5% at both alignments.

With default alignment, the qual-only case improved latency by
roughly 5-6%. With 64-byte alignment, however, the improvement
disappeared. This interaction reproduced in two independently
balanced measurement series.

PMU measurements showed increased frontend-bound activity and
DSB-to-MITE transition penalties in the aligned qual-only case.
However, there was no convincing increase in L1I misses or
instruction-fetch stalls. This suggests that code placement and
instruction delivery matter, but does not demonstrate I-cache
pollution.

An earlier mixed-workload run showed a small slowdown, but it
did not reproduce reliably in subsequent balanced experiments.
I therefore don't consider it evidence of a general regression.

Interestingly, even a SeqScan-only control showed changes in
frontend delivery metrics, despite not executing any of the new
IndexOnlyScan variants. This suggests that relocation of existing
executor code can matter independently of the additional variants.

I also reviewed the correctness aspects on both macOS ARM64 and
Linux x86-64, focusing on EPQ, runtime keys, rescans, and the
pg_assume() invariants. I found no correctness regression.

One useful gap emerged from that investigation: the existing tests
do not appear to exercise a correlated IndexOnlyScan within EPQ.

I've attached a small test-only isolation patch covering this path.
It uses a concurrent UPDATE and three rows to verify refreshed
runtime keys, NULL handling, qual rejection, and projection within
the same EPQ state. Instrumentation confirmed that v2 enters
ExecIndexOnlyScanEPQ(). The test passes on both unpatched master and
v2, including 140 repeated runs across -O2 and -O0 builds. I also
verified that it detects five deliberately injected faults involving
EPQ dispatch, runtime keys, quals, and projection.

The test does not depend on the specialization patch itself, so it
could be considered independently.

Overall, the results have addressed most of my earlier concerns.
The code-layout sensitivity is interesting, but I haven't found
evidence of a reproducible mixed-workload regression or a correctness
problem that would argue against this optimization.

I'd appreciate your thoughts on whether the attached EPQ isolation
test would be useful to include.

Thanks,
Haibo