| From: | Justin AnyhowStep <anyhowstep(at)hotmail(dot)com> |
|---|---|
| To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Dean Rasheed <dean(dot)a(dot)rasheed(at)gmail(dot)com> |
| Cc: | PostgreSQL mailing lists <pgsql-bugs(at)lists(dot)postgresql(dot)org> |
| Subject: | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
| Date: | 2020-03-01 02:59:38 |
| Message-ID: | SG2PR02MB4458A55D3751B606E571F84BDBE60@SG2PR02MB4458.apcprd02.prod.outlook.com |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-bugs |
> With this patch, all the examples originally posted return the correct
> results (calculated with bc). I'd be interested to know how the OP
> constructed these examples, and whether there were any that were off
> by more than 1 ULP.
> Yeah, that would be interesting.
I didn't do anything fancy. I'm learning how to write an arbitrary precision math library.
I couldn't come up with good test cases since so many numbers exist.
So, I sanity-checked my code by testing many random inputs against another library and pg, and compared the results.
Most of the time, the problem would be with my code.
But I found a few cases where it looked like pg was off.
I don't think I found any that were off by more than 1.
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Dean Rasheed | 2020-03-01 15:34:37 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |
| Previous Message | Tom Lane | 2020-02-29 19:12:39 | Re: BUG #16281: LN() function inaccurate at 1000th fractional digit |