> On Nov 11, 2026, Andrei Lepikhov wrote:
> > I think it's worth looking at the earlier discussion on this list.
> > 0001 is essentially the approach we discussed previously [1], which
> > concluded with 6bb6a62f3 - only using the product where the extended
> > stats backed it up. If you have any new arguments for re-visiting
> > that, please put them forward.
>
> Thanks for the reply. The PATCH 1/2 email does discuss 6bb6a62f3
> directly; its commit message says:
>
> Commit 6bb6a62f3 added estimate_multivariate_bucketsize(), which
> answers from a multivariate ndistinct statistics object when one
> exists; that function is left alone. The clauses it does not
> answer -- any table without such an object, which is the common
> case -- are estimated here instead, with estimate_num_groups()
>
> and the code comment in the patch repeats the same point ("extended
> statistics, when they exist, have already had their say ..."). But the
> sentence that matters is yours: whether the series earns a re-visit of
> that decision. Let me lay out the full picture.
>
> That thread started from the same symptom this series reports: with
> several hash clauses, the per-clause loop divides by the largest
> single-column ndistinct, hash join is priced out, and a slower merge
> join wins. The bare product proposed there died on Tomas's objection
> -- under correlation it prices buckets that do not exist -- and
> 6bb6a62f3 was the principled middle ground: use a better ndistinct
> estimate, but only where extended statistics back it up. 0001 keeps
> that stage intact and in priority. Mechanically, in
> final_cost_hashjoin() the statistics estimator runs first and folds its
> fraction into innerbucketsize; hash_join_joint_ndistinct() then sees
> only the clauses it returns uncovered; and the final min() treats every
> estimate, statistical or new alike, as a bound on the bucket, so the
> new estimate can only shrink the result, never enlarge it. The gap it
> fills is real: for the uncovered clauses -- tables without such
> statistics, the common case -- the status quo is the per-clause
> minimum, i.e. exactly the conservatism the thread began from.
>
> Since you ask for new arguments for re-visiting that remaining line,
> here is what I have:
>
> 1. Consistency within one join node. calc_joinrel_size_estimate()
> already prices the output rows of this very join under the
> independence assumption, while the per-clause loop prices the bucket
> by the largest single-column ndistinct. The same cost table thus
> holds two contradictory answers for the same hash key.
> estimate_num_groups() is the estimator the row count already comes
> from, so bucket and rows can no longer disagree: if independence is
> wrong for the bucket, it is already wrong for the rows the planner
> prints.
>
> 2. It is not the bare product Tomas objected to. estimate_num_groups()
> works from sampled per-column ndistinct, applies functional
> dependencies when a statistics object covers the columns, and bounds
> the result by the number of input rows. The stats_ext.out case this
> patch must update shows the difference: the hash side there (sb_2)
> has effectively independent keys (coprime moduli 49/51/73), and the
> input-row bound recovers the true joint ndistinct (~10^4) that the
> per-clause estimate cannot see. The old estimate overpriced that
> bucket about 137x (1/73 vs 1e-4), so the plan flip says the old
> estimate was wrong, not that the new one is optimistic. (The stale
> "-- Choose merge join" comments in that test will be fixed in v2.)
>
> 3. SEMI/ANTI joins are excluded, which the 2023 proposal did not do.
> Their probe charge is already scaled by inner_scan_frac, which
> describes the same duplicate factor, so re-pricing the bucket
> changes no work; it only moves such paths down the same cost table.
> Excluding them was forced by measurement: keeping them regressed a
> merge-semi-join query in the workload by 27%.
>
> On the motivating query the corrected bucket estimate turns a
> never-chosen hash join (cost 166,622,722) into the winning plan (cost
> 282,717; 1,946 ms vs 4,154 ms for the merge join it used to pick), and
> runs with and without the series stayed within the +/-2% run-to-run
> noise band.
>
> The exposure I do not have a principled answer for remains an inner
> side with fully correlated key columns and no statistics (the sb_1
> construction, x = y = z), where the estimate is optimistic by roughly
> the per-column product. My position is that consistency with the rows
> estimator is still the better bet there, because the planner already
> takes that bet for the join's row count. But if you see a shape where
> this clearly misleads plan choice, I will gladly add it as a test case
> and either bound or document the behavior.
>
> Thanks,
> He Kuang