Re: [PATCH] Fix overflow and underflow in regr_r2()

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: Chengpeng Yan <chengpeng_yan(at)outlook(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] Fix overflow and underflow in regr_r2()
Date: 2026-05-16 18:42:59
Message-ID: 1513654.1778956979@sss.pgh.pa.us
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
> On Sat, 16 May 2026 at 17:45, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> One simple change that might make things better is to compute
>> PG_RETURN_FLOAT8((Sy - Sx * (Sxy / Sxx)) / N);
>> on the theory that the sums of products are likely to both be large.

> Hmm, that isn't necessarily better.

True. We could do something like

(Sy - exp(log(Sx) + log(Sxy) - log(Sxx))) / N

but this'd require additional special-case code to deal with zero
or negative Sx and Sxy, so it's feeling rather tedious.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Dean Rasheed 2026-05-16 20:34:25 Re: [PATCH] Fix overflow and underflow in regr_r2()
Previous Message Dean Rasheed 2026-05-16 18:03:45 Re: [PATCH] Fix overflow and underflow in regr_r2()