Re: Fw:Re: Fw: gbt_var_consistent in contrib/btree_gist/btree_utils_var.c has internal-node type confusion on the <> strategy, bypassing exclusion constraints

Hi,

On Thu, 2 Jul 2026 at 22:24, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Ayush Tiwari <ayushtiwari(dot)slg01(at)gmail(dot)com> writes:
> > On Thu, 2 Jul 2026 at 20:08, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> None of this passes the smell test for me. If it's unsafe to
> >> call the type's f_eq function on a truncated key, how is it
> >> any safer to call the f_cmp function?
>
> > I think the difference is that for bit/varbit f_eq and f_cmp aren't the
> > same kind of function. f_eq is biteq (reads bit_len, does VARSIZE-8),
> > while f_cmp is byteacmp (plain byte-wise, only needs the varlena
> > header).
>
> Oh! You are right, the internal keys are *not* valid bit/varbit
> values, just bytea. The level of non-documentation in this code
> is staggering.
>
> So the actual API contract here is that f_eq etc. are to be used only
> on leaf keys, while f_cmp is to be used only on internal keys. It's
> not too surprising that whoever added NotEqual support didn't know
> that, it being documented nowhere.
>
> It kind of looks actually like there's an expectation that all
> internal keys are effectively bytea's; note the naming of
> gbt_bytea_pf_match, despite the fact that it's applied to all
> types under the purview of btree_utils_var.c.
>
> Anyway, I think part of our work here has got to be to raise the
> level of documentation of this code. I've had it with having
> to reverse-engineer details as fundamental as these.
>

I've added some documentation in code, patch attached.

Please help reviewing if it looks good or should we be adding
something more.

Regards,
Ayush