Skip site navigation (1) Skip section navigation (2)

Re: BUG in postgres mathematic

From: "Robert B(dot) Easter" <reaster(at)comptechnews(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Max Vaschenko <max(at)nino(dot)ru>, pgsql-bugs(at)postgresql(dot)org, Thomas Lockhart <lockhart(at)alumni(dot)caltech(dot)edu>
Subject: Re: BUG in postgres mathematic
Date: 2001-01-27 00:31:35
Message-ID: 0101261931350S.08820@comptechnews (view raw or flat)
Thread:
Lists: pgsql-bugs
On Friday 26 January 2001 18:07, Tom Lane wrote:
> Curiously, this change exposed what I take to be a platform dependency
> in the int8 regress test.  It was computing
> int8(float8(4567890123456789::int8)) and expecting to get back exactly
> 4567890123456789.  However, that value is 53 bits long and so there is
> no margin for error in a standard IEEE float8 value.  I find that at
> least on HP hardware, rint() treats the value as inexact and rounds to
> nearest even:
>
> regression=# select round(4567890123456788::float8) -
> 4567890123456780::float8; ?column?
> ----------
>         8
> (1 row)
>
> regression=# select round(4567890123456789::float8) -
> 4567890123456780::float8; ?column?
> ----------
>         8
> (1 row)
>
> regression=# select round(4567890123456790::float8) -
> 4567890123456780::float8; ?column?
> ----------
>        10
> (1 row)
>
> regression=#
>
> Whether this is a bug in rint or spec-compliant behavior is unclear, but
> I'll bet HP's hardware is not the only platform that behaves this way.
> Since I'm not eager to try to develop a new set of platform-specific
> int8 expected files at this late hour, I just diked out that test
> instead...

Here is what I get on Linux (PIII):

reaster=# select round(4567890123456788::float8) - 4567890123456780::float8;
 ?column?
----------
        8
(1 row)
 
reaster=# select round(4567890123456789::float8) - 4567890123456780::float8;
 ?column?
----------
        9
(1 row)
 
reaster=# select round(4567890123456790::float8) - 4567890123456780::float8;
 ?column?
----------
       10
(1 row)  

I'm not sure what the problem is either.  The PIII has an 80-bit FPU but not 
sure that matters.  When there is no exponent, maybe only 52 bits are really 
in the mantissa.  If you try rounding numbers <= 4503599627370495 (2^52 - 1), 
maybe you'll get expected results.  The hidden bit is 0.  Could be that round 
or rint (whatever it is) always makes the hidden bit 1 when I think it should 
only be 1 when the exponent is nonzero.  I'm no float expert! :)  Feel free 
to correct me.


-- 
-------- Robert B. Easter  reaster(at)comptechnews(dot)com ---------
-- CompTechNews Message Board http://www.comptechnews.com/ --
-- CompTechServ Tech Services http://www.comptechserv.com/ --
---------- http://www.comptechnews.com/~reaster/ ------------

In response to

pgsql-bugs by date

Next:From: Tom LaneDate: 2001-01-27 00:35:18
Subject: Re: select fails on indexed varchars.
Previous:From: Alex KrohnDate: 2001-01-27 00:03:49
Subject: Re: select fails on indexed varchars.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group