Re: intarray: fix an edge case int32 overflow bug

From: David Rowley <dgrowleyml(at)gmail(dot)com>
To: Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>
Cc: Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: intarray: fix an edge case int32 overflow bug
Date: 2026-01-04 06:28:46
Message-ID: CAApHDvp2f_1ecHx+R_s9QtJAHO5AWwOCop3cW5q=0vox+-E+=A@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Sun, 4 Jan 2026 at 16:20, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com> wrote:
> I noticed an int32 overflow problem in intarray’s compare_val_int4():
> ```
> /*
> * Comparison function for binary search in mcelem array.
> */
> static int
> compare_val_int4(const void *a, const void *b)
> {
> int32 key = *(int32 *) a;
> const Datum *t = (const Datum *) b;
>
> return key - DatumGetInt32(*t);
> }
> ```
>
> As this function is a bsearch comparator, it is supposed to return >0, =0 or <0. However this function uses subtraction with two int32 and returns an int, which may result in an overflow. Say, key is INT32_MAX and *t is -1, the return value will be negative due to overflow.

Nice find. Was that found by a static analyser or by eye?

I can take care of the overflow issue. I feel the test is a step too
far as it seems unlikely ever to be rebroken, but thanks for the
SQL-based test case to demonstrate the issue.

David

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message David Rowley 2026-01-04 07:35:18 Re: intarray: fix an edge case int32 overflow bug
Previous Message Pavel Stehule 2026-01-04 05:34:24 Re: [PATCH] psql: add size-based sorting options (O/o) for tables and indexes