| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> |
| Cc: | Chengpeng Yan <chengpeng_yan(at)outlook(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: [PATCH] Fix overflow and underflow in regr_r2() |
| Date: | 2026-05-16 16:45:16 |
| Message-ID: | 1506197.1778949916@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
BTW, on the principle of "where else did we make the same mistake",
I looked through the other aggregates using float8_regr_accum.
Most seem okay, but float8_regr_intercept does this:
PG_RETURN_FLOAT8((Sy - Sx * Sxy / Sxx) / N);
Seems to me that expression is also prone to internal
overflow/underflow. Underflow probably isn't a huge issue,
since the result will reduce to Sy/N which is likely to be good
enough. But can we do anything about overflow?
One simple change that might make things better is to compute
PG_RETURN_FLOAT8((Sy - Sx * (Sxy / Sxx)) / N);
on the theory that the sums of products are likely to both be large.
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Paul A Jungwirth | 2026-05-16 16:50:35 | Re: Enforce INSERT RLS checks for FOR PORTION OF leftovers? |
| Previous Message | Hüseyin Demir | 2026-05-16 16:37:31 | Re: Checkpoint replication slots later |