| From: | James Cloos <cloos(at)jhcloos(dot)com> |
|---|---|
| To: | Tom Duffey <tduffey(at)trillitech(dot)com> |
| Cc: | pgsql-general(at)postgresql(dot)org |
| Subject: | Re: Floating point error |
| Date: | 2013-03-01 01:05:42 |
| Message-ID: | m3r4k07wu8.fsf@carbon.jhcloos.org |
| Views: | Raw Message | Whole Thread | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general pgsql-hackers |
>>>>> "TD" == Tom Duffey <tduffey(at)trillitech(dot)com> writes:
TD> Riddle me this. I have a database column of type "real" that gets
TD> mapped to a Java field of type double via JDBC. ...
TD> - Selecting values from both test and production DBs using psql
TD> shows "10.3885" as the value
TD> - The Java app on production shows "10.3884573" while the test app
TD> shows "10.3885"
I suspect the issue is that psql(1) and whatever java method you use to
convert the floats to text choose different rounding.
By default, it seems that psql(1) uses something like printf("%.4f",...)
whereas your java app calls a routing which works more like "%.7f".
(The wire format for floats is the same as they are stored, not a text
representation thereof.)
-JimC
--
James Cloos <cloos(at)jhcloos(dot)com> OpenPGP: 1024D/ED7DAEA6
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Duffey | 2013-03-01 01:55:15 | Re: Floating point error |
| Previous Message | Chris Hanks | 2013-03-01 00:19:03 | Re: Poor performance when using a window function in a view |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Tom Duffey | 2013-03-01 01:55:15 | Re: Floating point error |
| Previous Message | Steve Singer | 2013-03-01 00:53:48 | Re: json generation enhancements |