| From: | David Rowley <dgrowleyml(at)gmail(dot)com> |
|---|---|
| To: | Michael Paquier <michael(at)paquier(dot)xyz> |
| Cc: | Andrei Lepikhov <lepihov(at)gmail(dot)com>, Chao Li <li(dot)evan(dot)chao(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>, Andres Freund <andres(at)anarazel(dot)de>, Nathan Bossart <nathandbossart(at)gmail(dot)com> |
| Subject: | Re: SUM(int2)/SUM(int4) do not detect overflow of the int8 accumulator |
| Date: | 2026-09-04 08:55:07 |
| Message-ID: | CAApHDvp-3v1OWbo7RavBf37=MjNScqhj4vyAyHOUDY51nGUnog@mail.gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Fri, 4 Sept 2026 at 16:59, Michael Paquier <michael(at)paquier(dot)xyz> wrote:
> Any thoughts or comments from others?
I think if we don't add overflow error checking for sum(int2) and
sum(int4) today, we'll need to do it at some point in the future. I
suspect we've only gotten away with it for this long, not because
nobody aggregates 4+ billion rows, but because the values being
aggregated are unlikely to be large enough to cause the overflow.
Since Andrei has demonstrated that it's possible to hit that limit
with a non-parallel query in less than 5 minutes, albeit that is
passing INT_MAX (the most extreme case), it might be worth adding the
checks. I was surprised that it only took 5 mins to do 4 billion rows,
especially with generate_series. It's probably just a matter of time
before someone discovers this with a real-world case out in the wild.
If there's a measurable performance regression from adding the
overflow checks, does the attached buy enough of it back? I couldn't
really measure much of a performance difference from it on my Zen2
machine, so I didn't try with the overflow patch.
The patch adds PG_RETURN_INPUT(n) to avoid some of the branching in
int4_sum() so that it immediately returns the aggstate when the value
being aggregated is null. With my compiler, it cut int4_sum from 18
down to 16 instructions.
David
| Attachment | Content-Type | Size |
|---|---|---|
| sum_int4_speedup.txt | text/plain | 1.7 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tender Wang | 2026-09-04 09:30:00 | Re: Assert failure in try_nestloop_path() |
| Previous Message | Nisha Moond | 2026-09-04 08:41:40 | Re: Introduce XID age based replication slot invalidation |