Re: Let an ordering index scan hand its ORDER BY value to the target list

From: Greg Burd <greg(at)burd(dot)me>
To: Oleg Bartunov <obartunov(at)postgrespro(dot)ru>
Cc: Matthias van de Meent <boekewurm+postgres(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Chris Cleveland <ccleveland(at)dieselpoint(dot)com>
Subject: Re: Let an ordering index scan hand its ORDER BY value to the target list
Date: 2026-10-15 19:02:40
Message-ID: WHeHYWrzaBPn3qIH4LB2vbrV3PPqg3Opy1JmZf657Nq0FnZ_7NAT2Jnca2Jlju8ZB7BUEK5ra8UVcAodH32Zilckw7XIf5NfkbIA0McVDas=@burd.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Monday, October 12th, 2026 at 8:34 AM, Oleg Bartunov <obartunov(at)postgrespro(dot)ru> wrote:

> Hi Greg,

Hey Oleg!

Thank you for taking the time to try this, and for writing up what you
found. So much of GIN and kNN search came from you and your colleagues
that it means a lot to hear the changes make sense from your side.
Until now only GiST and SP-GiST implemented amcanreturnorderby, with
functions I wrote to be slow, so yours is the first test against a
ranking AM. I'm glad the costing in 0002 helped too.

> I tested your patch with a ranked GIN prototype - essentially GIN with RUM-like ordering, where the index scan computes the BM25 score and returns tuples in score order.

If you can share the prototype, or a script that builds a table and
runs your queries, I'd like to run it before I post each version.

> This is a very natural use case for amcanreturnorderby: the index already has the exact score, so there is no reason to calculate the same expression again above the index scan.

Yes, it is my pg_fts[1] bm25 extension that highlighted this issue for me.

> With the patch, the score produced by the index is reused directly. On my tests this removes the redundant score calculation and gives about 1.7 - 2.2x speedup for full scans, and about 2x on larger toasted documents.

Yes, that's what I was shooting for.

> The costing change is useful too. Without it, the planner can still charge the ordered index scan for evaluating an expression which the index has already evaluated, and may choose a Seq Scan + Sort instead. With the patch this distinction is visible to the planner.
> One remaining issue on my side is that the cost of BM25 scoring depends heavily on document size/detoasting, so a single static function COST is not enough. But that seems like a separate costing problem.
> So at least for ranked GIN/RUM-style access methods, this patch provides exactly the missing interface between the index AM and executor.

You're right that one static COST can't follow document size. I
haven't tried this yet, so take it as a question: could the AM's
amcostestimate scale the scoring cost by the column's average width,
or could a support function answer SupportRequestCost? I agree it's
a separate problem from this series either way.

> Thanks, Oleg

If you have time to review the series, I'd be grateful to see
your name on the commitfest entry [2]. Please don't feel obliged.

best.

-greg

[1] https://github.com/gburd/pg_fts
[2] https://commitfest.postgresql.org/patch/7392/

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Sami Imseih 2026-10-15 19:03:21 Re: [PATCH] Doc fix for autovacuum_max_parallel_workers
Previous Message Daniel Gustafsson 2026-10-15 19:02:16 Re: Serverside SNI support in libpq