From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Gregory Stark <stark(at)enterprisedb(dot)com> |
Cc: | "Filip Krska" <filip(dot)krska(at)comstar(dot)cz>, pgsql-bugs(at)postgresql(dot)org, "Bruce Momjian" <bruce(at)momjian(dot)us> |
Subject: | Re: BUG #3387: mod on non-integer returns bad result |
Date: | 2007-06-15 17:47:33 |
Message-ID: | 6531.1181929653@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> In the case reported div_var was getting 70/70 = 0.99999. Which is
> really just wrong.
Agreed, but I think your proposed patch is just a band-aid. The real
problem with div_var is that it generates inaccurate output digits at
all --- it's assuming that computing a few guard digits before rounding
will suffice to ensure that all the delivered digits are correct, but
I think there will always be corner cases where it fails.
I just blew the dust off my old copy of Knuth vol 2, and see that his
algorithm for multi-precision division generates output digits that are
correct to start with (or at least he never needs to revisit a digit
after moving on to the next). ISTM we should go over to an approach
like that. For the truncated-output case the result will be exact,
and for the rounded-output case you generate exactly one guard digit
to see if it's >= base/2.
regards, tom lane
From | Date | Subject | |
---|---|---|---|
Next Message | Alvaro | 2007-06-17 21:45:28 | BUG #3390: initdb fails on install |
Previous Message | Gregory Stark | 2007-06-15 16:54:46 | Re: BUG #3387: mod on non-integer returns bad result |