Re: BUG #3387: mod on non-integer returns bad result

From: Bruce Momjian <bruce(at)momjian(dot)us>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Gregory Stark <stark(at)enterprisedb(dot)com>, Filip Krska <filip(dot)krska(at)comstar(dot)cz>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #3387: mod on non-integer returns bad result
Date: 2007-07-17 04:54:51
Message-ID: 200707170454.l6H4spR05607@momjian.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs


This has been saved for the 8.4 release:

http://momjian.postgresql.org/cgi-bin/pgpatches_hold

---------------------------------------------------------------------------

Tom Lane wrote:
> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> > In the case reported div_var was getting 70/70 = 0.99999. Which is
> > really just wrong.
>
> Agreed, but I think your proposed patch is just a band-aid. The real
> problem with div_var is that it generates inaccurate output digits at
> all --- it's assuming that computing a few guard digits before rounding
> will suffice to ensure that all the delivered digits are correct, but
> I think there will always be corner cases where it fails.
>
> I just blew the dust off my old copy of Knuth vol 2, and see that his
> algorithm for multi-precision division generates output digits that are
> correct to start with (or at least he never needs to revisit a digit
> after moving on to the next). ISTM we should go over to an approach
> like that. For the truncated-output case the result will be exact,
> and for the rounded-output case you generate exactly one guard digit
> to see if it's >= base/2.
>
> regards, tom lane

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://www.enterprisedb.com

+ If your life is a hard drive, Christ can be your backup. +

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Hiroshi Saito 2007-07-17 06:23:10 Re: BUG #3453: Error on COPY TO/FROM 'non-ascii-path'
Previous Message ITAGAKI Takahiro 2007-07-17 04:24:08 BUG #3453: Error on COPY TO/FROM 'non-ascii-path'