Re: BUG #16281: LN() function inaccurate at 1000th fractional digit

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com>
Cc: anyhowstep(at)hotmail(dot)com, PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org>
Subject: Re: BUG #16281: LN() function inaccurate at 1000th fractional digit
Date: 2020-02-29 19:12:39
Message-ID: 13048.1583003559@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> writes:
> Looking more closely at ln_var(), it seems that there was an oversight
> related to the way that it computes the local_rscale for the Taylor
> series expansion --- it fails to account for the fact that the result
> is multiplied by fact (2^(nsqrt+1), where nsqrt is the number of
> square roots performed in the range reduction phase, which in practice
> is at most 22).
> Since 2^22 has 7 decimal digits, multiplying by 2^22 almost entirely
> wipes out the 8-digit safety margin used in the Taylor series
> expansion. The attached patch corrects that.

Looks sane to me.

> With this patch, all the examples originally posted return the correct
> results (calculated with bc). I'd be interested to know how the OP
> constructed these examples, and whether there were any that were off
> by more than 1 ULP.

Yeah, that would be interesting.

regards, tom lane

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Justin AnyhowStep 2020-03-01 02:59:38 Re: BUG #16281: LN() function inaccurate at 1000th fractional digit
Previous Message Dean Rasheed 2020-02-29 17:16:02 Re: BUG #16281: LN() function inaccurate at 1000th fractional digit