Re: Possible rounding error of large float values?

From: Stephan Szabo <sszabo(at)megazone(dot)bigpanda(dot)com>
To: Iain <iain(at)mst(dot)co(dot)jp>
Cc: Adam Lancaster <alancaster(at)preventsys(dot)com>, pgsql-sql(at)postgresql(dot)org
Subject: Re: Possible rounding error of large float values?
Date: 2004-08-25 01:41:37
Message-ID: 20040824183612.M67501@megazone.bigpanda.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

On Wed, 25 Aug 2004, Iain wrote:

>
> # select 9223372036854775807 = 9223372036854775807::float;
> ?column?
> ----------
> t
> (1 row)

> This and the fact that it's still possible to find the row using the
> original value would seem to indicate that the rounding is just a display
> artifact..

I don't think that's true since the number ending in 807 is equal to a
version ending in 808.

sszabo=# select 9223372036854775807 = 9223372036854775808::float;
?column?
----------
t
(1 row)

(In this next one I'm casting both because I want to choose the types of
both literals)

sszabo=# select 9223372036854775807::float =
9223372036854775808::numeric;
?column?
----------
t
(1 row)

In response to

Responses

Browse pgsql-sql by date

  From Date Subject
Next Message Iain 2004-08-25 02:48:11 Re: Possible rounding error of large float values?
Previous Message Iain 2004-08-25 01:32:29 Re: Possible rounding error of large float values?