Possible rounding error of large float values?

From: "Adam Lancaster" <alancaster(at)preventsys(dot)com>
To: <pgsql-sql(at)postgresql(dot)org>
Subject: Possible rounding error of large float values?
Date: 2004-08-17 17:21:22
Message-ID: 8758F8D58219684FAB0239EE8967048A010807@calculon.preventsys.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-sql

When setting a float column to this value:

9223372036854775807

It gets selected out as:

9.22337203685478E18

Which appears to be rounded. When we cast it to numeric type we get:

9223372036854780000

Which also is rounded. It is still possible to find the row using the
original value

select * from <table> where <col> = 9223372036854775807

Is this expected behavior?

Thanks,
Adam

Responses

Browse pgsql-sql by date

  From Date Subject
Next Message Bruno Wolff III 2004-08-17 17:31:50 Re: SELECT MAX(c) FROM (SELECT ... FOR UPDATE) AS foo
Previous Message Jean-Luc Lachance 2004-08-17 15:54:14 Re: SELECT MAX(c) FROM (SELECT ... FOR UPDATE) AS foo