Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Philip Warner <pjw(at)rhyme(dot)com(dot)au>
Cc: "Romanenko Mikhail" <mikhail(at)angg(dot)ru>, pgsql-general(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2
Date: 2000-08-07 16:11:48
Message-ID: 6426.965664708@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

Philip Warner <pjw(at)rhyme(dot)com(dot)au> writes:
> pjw=# select float8(float4(10.1));
> float8
> ------------------
> 10.1000003814697
> (1 row)

> I would have expected the latter to be at worst 10.10000000000000 +/-
> .00000000000001.

float4 is good to about 7 decimal digits (24 mantissa bits) on
IEEE-standard machines. Thus the above result is actually closer
than you have any right to expect.

Don't they teach people about float arithmetic in CS 101 anymore?

regards, tom lane

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Thomas Lockhart 2000-08-07 16:12:59 Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2
Previous Message Thomas Lockhart 2000-08-07 16:04:54 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Lockhart 2000-08-07 16:12:59 Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2
Previous Message Thomas Lockhart 2000-08-07 16:04:54 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2