Re: BUG #19340: Wrong result from CORR() function

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: Oleg Ivanov <o15611(at)gmail(dot)com>, Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #19340: Wrong result from CORR() function
Date: 2025-12-06 18:18:43
Message-ID: 2319120.1765045123@sss.pgh.pa.us
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
> The first result is required by the SQL standard, which says first
> that if N * Sxx = Sx * Sx, then the result is NULL, and similarly for
> y. So that rule should take precedence over any rule for NaNs.

> The second result can be justified by the IEEE rules for NaNs,
> according to which NaN does not equal any other number, including
> itself, and so an all-NaN column is not all equal (and it doesn't
> satisfy "N * Sxx = Sx * Sx" either). So the SQL standard all-the-same
> rule doesn't apply to the second query, and the standard computation
> yields NaN.

Right, that's pretty much the thinking I've ended up on.

> If we're happy with those decisions, then I think this comment should
> be updated:

Here's a v3 with another try at that comment, and the other points
addressed. Also now with a draft commit message. I credited you
as co-author since so much of this is your ideas.

regards, tom lane

Attachment Content-Type Size
v3-0001-Handle-constant-inputs-to-corr-and-related-aggreg.patch text/x-diff 24.6 KB

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2025-12-06 21:04:48 Re: BUG #19345: MemoryContextSizeFailure after upgrade 14.11 to 17.7 in stored procedure
Previous Message Dean Rasheed 2025-12-06 11:51:20 Re: BUG #19340: Wrong result from CORR() function