Re: Redesign the EXPLAIN metric ‘Removed by Join Filter’?

From: Andrei Lepikhov <lepihov(at)gmail(dot)com>
To: pgsql-hackers(at)lists(dot)postgresql(dot)org
Cc: Nikolay Samokhvalov <nik(at)postgres(dot)ai>, Lukas Fittl <lukas(at)fittl(dot)com>, Tomas Vondra <tomas(at)vondra(dot)me>
Subject: Re: Redesign the EXPLAIN metric ‘Removed by Join Filter’?
Date: 2026-11-10 16:06:28
Message-ID: cd106b55-3ab5-45a6-8ba9-eb59b272ae20@gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 25/09/2026 16:53, Andrei Lepikhov wrote:
> Having this metric, Postgres would enable DBAs and extensions to search for
> beneficial indexes - not only for scan purposes as now, but for the JOIN too.

Hi,

After some analysis, I found that counting unmatched outer rows is almost free:
join nodes already track whether the current outer tuple has found a partner, so
it is enough to count matched outer tuples and subtract that from the outer
child's row count. See 0001 in the attachment: the executor changes are a few
lines per join node. The 0002 (mostly auto-generated) contains docs and tests.

This new number does not depend on the join method. With the same outer side,
NestLoop, HashJoin and MergeJoin count the same matched rows.

The idea is close to 'Rows Removed by Filter' on a scan. I see several uses:
1. Find cases where a parameterised NestLoop could beat other join types - in
other words, a missing index on the join key.
2. Spot an inefficient join order that carries many tuples further up the plan
than necessary.
3. Expose the hidden work of hash and merge clauses, which may go through tons
of tuples to find a few matching ones.
4. Possibly, find cases where a Bloom filter could speed up the query.

One corner case:
Hash Right Semi Join reports nothing: it skips already matched inner tuples
before checking the join qual, so it cannot tell which outer rows have a
partner. The inner side is not covered yet.

Thoughts?

--
regards, Andrei Lepikhov,
pgEdge

Attachment Content-Type Size
0001-Make-EXPLAIN-ANALYZE-report-the-number-of-unmatch.patch text/plain 7.5 KB
0002-Add-docs-and-tests-for-EXPLAIN-s-Outer-Rows-Unmat.patch text/plain 49.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Matheus Alcantara 2026-11-10 16:30:32 Re: [PATCH] doc: Fix description of relative extension "directory" setting
Previous Message Greg Burd 2026-11-10 16:01:40 Re: configure picks a clang that doesn't match LLVM_CONFIG