| From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
|---|---|
| To: | Philip Warner <pjw(at)rhyme(dot)com(dot)au> |
| Cc: | "Romanenko Mikhail" <mikhail(at)angg(dot)ru>, pgsql-general(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org |
| Subject: | Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 |
| Date: | 2000-08-07 16:11:48 |
| Message-ID: | 6426.965664708@sss.pgh.pa.us |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-general pgsql-hackers |
Philip Warner <pjw(at)rhyme(dot)com(dot)au> writes:
> pjw=# select float8(float4(10.1));
> float8
> ------------------
> 10.1000003814697
> (1 row)
> I would have expected the latter to be at worst 10.10000000000000 +/-
> .00000000000001.
float4 is good to about 7 decimal digits (24 mantissa bits) on
IEEE-standard machines. Thus the above result is actually closer
than you have any right to expect.
Don't they teach people about float arithmetic in CS 101 anymore?
regards, tom lane
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thomas Lockhart | 2000-08-07 16:12:59 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
| Previous Message | Thomas Lockhart | 2000-08-07 16:04:54 | Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Thomas Lockhart | 2000-08-07 16:12:59 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
| Previous Message | Thomas Lockhart | 2000-08-07 16:04:54 | Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 |