Re: Exponentiation confusion

On Thu, 13 Oct 2022 at 18:17, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> I'm inclined to think that we should push the responsibility for choosing
> its rscale into power_var_int(), because internally that already does
> estimate the result weight, so with a little code re-ordering we won't
> need duplicative estimates. Don't have time to work on that right now
> though ... Dean, are you interested in fixing this?
>

OK, I'll take a look.

The most obvious thing to do is to try to make power_var_int() choose
the same result rscale as power_var() so that the results are
consistent regardless of whether the exponent is an integer.

It's worth noting, however, that that will cause in a *reduction* in
the output rscale rather than an increase in some cases, since the
power_var_int() code path currently always chooses an rscale of at
least 16, whereas the other code path in power_var() uses the rscales
of the 2 inputs, and produces a minimum of 16 significant digits,
rather than 16 digits after the decimal point. For example:

select power(5.678, 18.00000001::numeric);
power
-------------------------
37628507689498.14987457
(1 row)

select power(5.678, 18::numeric);
power
---------------------------------
37628507036041.8454541428979479
(1 row)

Regards,
Dean