Skip site navigation (1) Skip section navigation (2)

Re: BUG #3387: mod on non-integer returns bad result

From: Gregory Stark <stark(at)enterprisedb(dot)com>
To: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: "Filip Krska" <filip(dot)krska(at)comstar(dot)cz>, <pgsql-bugs(at)postgresql(dot)org>, "Bruce Momjian" <bruce(at)momjian(dot)us>
Subject: Re: BUG #3387: mod on non-integer returns bad result
Date: 2007-06-15 16:54:46
Message-ID: 87r6odb1k9.fsf@oxford.xeocode.com (view raw or flat)
Thread:
Lists: pgsql-bugs
"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>> The source of the problem is the floating point arithmetic which is used to do
>> the individual steps in the long division.
>
> I don't think so.  The ultimate source of the problem is that div_var
> can only report a finite number of digits.

In the case reported div_var was getting 70/70 = 0.99999. Which is really just
wrong. The only reason was because 1.0/70 isn't representable so (1.0/70) * 70
is slightly more than 1 which ... div_var really ought not have any trouble
representing an integer even in its finite number of digits.

-- 
  Gregory Stark
  EnterpriseDB          http://www.enterprisedb.com


In response to

Responses

pgsql-bugs by date

Next:From: Tom LaneDate: 2007-06-15 17:47:33
Subject: Re: BUG #3387: mod on non-integer returns bad result
Previous:From: Gregory StarkDate: 2007-06-15 16:35:16
Subject: Re: BUG #3387: mod on non-integer returns bad result

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group