Re: BUG #19615: COVAR_POP / COVAR_SAMP / REGR_SXY return 0.0 instead of NaN

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Andrey Rachitskiy <pl0h0yp1(at)gmail(dot)com>
Cc: feasiblechart(at)gmail(dot)com, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #19615: COVAR_POP / COVAR_SAMP / REGR_SXY return 0.0 instead of NaN
Date: 2026-08-11 15:56:39
Message-ID: 2395189.1786463799@sss.pgh.pa.us
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Andrey Rachitskiy <pl0h0yp1(at)gmail(dot)com> writes:
> With a constant X and Inf arriving later in Y, commonX stays finite,
> so that update is skipped and Sxy remains 0. Before the commit, Sxy
> was always updated. With constant X, tmpX is ~0, so the product
> 0*Inf (or a tiny roundoff times Inf) produced NaN under IEEE rules.
> Inf in the first row still yields NaN, because the older first-input
> path from 33dd9bb3b0a is intact and forces Sxy to NaN up front.
> The new short-circuit never got the matching Inf/NaN handling.

Yeah, I just arrived at pretty much the same conclusion. We get
Inf/NaN handling right for Sxx and Syy, but not for the cross-product
Sxy.

regards, tom lane

In response to

Browse pgsql-bugs by date

  From Date Subject
Previous Message Andrey Rachitskiy 2026-08-11 15:24:19 Re: BUG #19615: COVAR_POP / COVAR_SAMP / REGR_SXY return 0.0 instead of NaN