From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Erik Wienhold <ewie(at)ewie(dot)name> |
Cc: | Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org> |
Subject: | Re: Exponentiation confusion |
Date: | 2022-10-13 17:16:58 |
Message-ID: | 979360.1665681418@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers |
Erik Wienhold <ewie(at)ewie(dot)name> writes:
> On 13/10/2022 18:20 CEST Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> wrote:
>> select power(10, -18::numeric);
>> power
>> --------------------
>> 0.0000000000000000
>>
>> Why is the cast throwing off the result?
> Calling power(numeric, numeric) is what I expect in that case instead of
> downcasting the exponent argument to double precision, thus losing precision.
An inexact result isn't surprising, but it shouldn't be *that* inexact.
It looks to me like numeric.c's power_var_int() code path is setting the
result rscale without considering the possibility that the result will
have negative weight (i.e. be less than one). The main code path in
power_var() does adjust for that, so for example
regression=# select power(10, -18.00000001::numeric);
power
-------------------------------------
0.000000000000000000999999976974149
(1 row)
but with an exact-integer exponent, not so much --- you just get 16 digits
which isn't enough.
I'm inclined to think that we should push the responsibility for choosing
its rscale into power_var_int(), because internally that already does
estimate the result weight, so with a little code re-ordering we won't
need duplicative estimates. Don't have time to work on that right now
though ... Dean, are you interested in fixing this?
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Thomas Kellerer | 2022-10-13 18:54:23 | pg_upgrade to 15 fails on Windows because of xml_is_well_formed() |
Previous Message | Peter J. Holzer | 2022-10-13 17:05:27 | Re: Exponentiation confusion |
From | Date | Subject | |
---|---|---|---|
Next Message | Melanie Plageman | 2022-10-13 17:29:32 | Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?) |
Previous Message | Peter J. Holzer | 2022-10-13 17:05:27 | Re: Exponentiation confusion |