Re: BUG #19340: Wrong result from CORR() function

From: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Oleg Ivanov <o15611(at)gmail(dot)com>, Laurenz Albe <laurenz(dot)albe(at)cybertec(dot)at>, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #19340: Wrong result from CORR() function
Date: 2025-12-02 23:03:45
Message-ID: CAEZATCUo6skdTfRP1XM1sbMveRo1PgpjzOMZEps69Bbz6-OnKA@mail.gmail.com
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On Tue, 2 Dec 2025 at 20:26, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> I experimented with the attached patch, which is very incomplete;
> I just carried it far enough to be able to run performance checks on
> the modified code, and so all the binary statistics aggregates except
> corr() are broken. I observe about 2% slowdown on this test case:

I played around with having just 2 extra array elements, constX and
constY equal to the common value if all the values are the same, and
NaN otherwise. For me, that was slightly faster, which I put down to
floating point comparison being faster than converting back and forth
between floating point and integer. Either way, it seems like a
difference that no one is likely to notice.

Doing it that way does lead to one difference though: all-NaN inputs
leads to a NaN result, whereas your patch produces NULL for that case.

Regards,
Dean

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2025-12-02 23:24:20 Re: BUG #19340: Wrong result from CORR() function
Previous Message Laurenz Albe 2025-12-02 21:44:53 Re: BUG #19341: REPLACE() fails to match final character when using nondeterministic ICU collation