Skip site navigation (1) Skip section navigation (2)

Re: float conversion / presentation problem ?

From: John R Pierce <pierce(at)hogranch(dot)com>
To:
Cc: weigelt(at)metux(dot)de, pgsql-bugs(at)postgresql(dot)org
Subject: Re: float conversion / presentation problem ?
Date: 2005-04-07 04:40:26
Message-ID: 4254B9BA.60008@hogranch.com (view raw or flat)
Thread:
Lists: pgsql-bugs
Tom Lane wrote:
> Enrico Weigelt <weigelt(at)metux(dot)de> writes:
> 
>>i've noticed an problem with conversion from real to float - 
>>this produces ugly values. (in fact it seems to be real->float8)
> 
> 
> You do realize that float4/real is only good to six decimal places
> on most platforms?
> 
> You can take the issue up with the glibc boys if you like, but
> I think you'll get laughed off.

and importantly, are stored as binary fractions.  0.10000 decimal is a 
repeating fraction in binary, its like .1100110011.... x 2^-2 or something as a 
FLOAT.   Float4 has like 22 bits of significants for the mantissa, while Float8 
has like 51 bits or something (this is from rusty memory, so I could have 
slipped a few bits here).

In response to

Responses

pgsql-bugs by date

Next:From: Tom LaneDate: 2005-04-07 05:02:50
Subject: Re: float conversion / presentation problem ?
Previous:From: Tom LaneDate: 2005-04-07 04:23:25
Subject: Re: float conversion / presentation problem ?

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group