Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2

From: Thomas Lockhart <lockhart(at)alumni(dot)caltech(dot)edu>
To: Philip Warner <pjw(at)rhyme(dot)com(dot)au>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Romanenko Mikhail <mikhail(at)angg(dot)ru>, pgsql-general(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2
Date: 2000-08-07 16:12:59
Message-ID: 398EE00B.772F181E@alumni.caltech.edu
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers

> I would have expected the latter to be at worst 10.10000000000000 +/-
> .00000000000001.
> Am I missing something?

Well, yes :)

10.1 can't be represented exactly, so the float8 representation has bits
set way down at the low end of the mantissa. When converting to float4
those low bits get rounded up or down into the lowest bit of the float4
representation. At that point, you have lost knowledge that this ever
was supposed to be *exactly* 10.1. And when converting back to float8,
that float4 low bit becomes a middle-range bit in the float8
representation, with all the bits underneath that zeroed.

Back in the old days, before printf() implementations settled down, you
would be reminded of this any time you did anything, since just
assigning 10.1 and then printing it out would give you some goofy
10.099999999998 or 10.10000000001 (don't count the number of digits here
too closely, they are only qualitatively correct).

- Thomas

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Philip Warner 2000-08-07 16:15:45 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2
Previous Message Tom Lane 2000-08-07 16:11:48 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2

Browse pgsql-hackers by date

  From Date Subject
Next Message Philip Warner 2000-08-07 16:15:45 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2
Previous Message Tom Lane 2000-08-07 16:11:48 Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2