Re: Exponentiation confusion

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Erik Wienhold <ewie(at)ewie(dot)name>
Cc: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>, Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com>, pgsql-general <pgsql-general(at)postgresql(dot)org>
Subject: Re: Exponentiation confusion
Date: 2022-10-13 17:16:58
Message-ID: 979360.1665681418@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Erik Wienhold <ewie(at)ewie(dot)name> writes:
> On 13/10/2022 18:20 CEST Adrian Klaver <adrian(dot)klaver(at)aklaver(dot)com> wrote:
>> select power(10, -18::numeric);
>> power
>> --------------------
>> 0.0000000000000000
>>
>> Why is the cast throwing off the result?

> Calling power(numeric, numeric) is what I expect in that case instead of
> downcasting the exponent argument to double precision, thus losing precision.

An inexact result isn't surprising, but it shouldn't be *that* inexact.
It looks to me like numeric.c's power_var_int() code path is setting the
result rscale without considering the possibility that the result will
have negative weight (i.e. be less than one). The main code path in
power_var() does adjust for that, so for example

regression=# select power(10, -18.00000001::numeric);
power
-------------------------------------
0.000000000000000000999999976974149
(1 row)

but with an exact-integer exponent, not so much --- you just get 16 digits
which isn't enough.

I'm inclined to think that we should push the responsibility for choosing
its rscale into power_var_int(), because internally that already does
estimate the result weight, so with a little code re-ordering we won't
need duplicative estimates. Don't have time to work on that right now
though ... Dean, are you interested in fixing this?

regards, tom lane

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Thomas Kellerer 2022-10-13 18:54:23 pg_upgrade to 15 fails on Windows because of xml_is_well_formed()
Previous Message Peter J. Holzer 2022-10-13 17:05:27 Re: Exponentiation confusion

Browse pgsql-hackers by date

  From Date Subject
Next Message Melanie Plageman 2022-10-13 17:29:32 Re: pg_stat_bgwriter.buffers_backend is pretty meaningless (and more?)
Previous Message Peter J. Holzer 2022-10-13 17:05:27 Re: Exponentiation confusion