Re: Fw:Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign

Hi,

On Sat, 13 Jun 2026 at 09:20, 王跃林 <violin0613(at)tju(dot)edu(dot)cn> wrote:

> 主题：Re: Fw: ltree_compare in contrib/ltree/ltree_op.c overflows int32 on deep ltree comparisons, returning the wrong sign
> On Mon, May 25, 2026 at 10:58:05PM +0800, violin0613(at)tju(dot)edu(dot)cn wrote:
> > PoC
> >
> > File vuln_001.sql
> >
> > CREATE EXTENSION IF NOT EXISTS ltree;
> > â
> > WITH s AS (SELECT 'a'::ltree AS v),
> > l AS (SELECT (repeat('a.', 19999) || 'a')::ltree AS v)
> > SELECT (l.v > s.v) AS long_gt_short_expected_true,
> > (l.v < s.v) AS long_lt_short_expected_false
> > FROM s, l;
> >
> > Process
> >
> > psql -h /tmp -p 36265 -U postgres -f vuln_001.sql
> >
> > Results
> >
> > long_gt_short_expected_true | long_lt_short_expected_false
> > -----------------------------+------------------------------
> > f | t
> >
> > Both columns are inverted. long > short returned false, long < short
> > returned true.
>
>
This looks like a classic case of integer overflow that's
happening in ltree_compare function in ltree_op.c.

return (al->len - bl->len) * 10 * (an + 1);
return res * 10 * (an + 1);
return (a->numlevel - b->numlevel) * 10 * (an + 1);

I think the calculation should be done as int64, something of this sort:

int64 v = (int64) (al->len - bl->len) * 10 * (an + 1);
if (v > PG_INT32_MAX) return PG_INT32_MAX;
if (v < PG_INT32_MIN) return PG_INT32_MIN;
return (int) v;

And needed to adjust the ltree_penalty function too.

Attached is a draft patch for this, I guess we can add a helper
function too for the above conversion.

Thoughts?

Regards,
Ayush