From: | Philip Warner <pjw(at)rhyme(dot)com(dot)au> |
---|---|
To: | Thomas Lockhart <lockhart(at)alumni(dot)caltech(dot)edu> |
Cc: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Romanenko Mikhail <mikhail(at)angg(dot)ru>, pgsql-general(at)postgresql(dot)org, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
Date: | 2000-08-07 16:28:37 |
Message-ID: | 3.0.5.32.20000808022837.02816100@mail.rhyme.com.au |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general pgsql-hackers |
At 16:12 7/08/00 +0000, Thomas Lockhart wrote:
>> I would have expected the latter to be at worst 10.10000000000000 +/-
>> .00000000000001.
>> Am I missing something?
>
>10.1 can't be represented exactly, so the float8 representation has bits
>set way down at the low end of the mantissa. When converting to float4
>those low bits get rounded up or down into the lowest bit of the float4
>representation. At that point, you have lost knowledge that this ever
>was supposed to be *exactly* 10.1. And when converting back to float8,
>that float4 low bit becomes a middle-range bit in the float8
>representation, with all the bits underneath that zeroed.
>
Now I understand, but it doesn't quite make sense given what was displayed.
The float4 value is *displayed* as 10.1, not 10.1000001, so I had assumed
that there was a level of either accuracy or display rouding happening.
When this value is converted to float8, I hoped that the result would be
the same as:
Cast( Cast(f4val as varchar(32)) as float8)
Maybe this hope is naieve, but it it a lot more useful than the current
situation. But now that I understand what is happening, I see that (short
of varchar conversions!), it is probably quite hard to do since we can't
tell the 'correct' value.
----------------------------------------------------------------
Philip Warner | __---_____
Albatross Consulting Pty. Ltd. |----/ - \
(A.C.N. 008 659 498) | /(@) ______---_
Tel: (+61) 0500 83 82 81 | _________ \
Fax: (+61) 0500 83 82 82 | ___________ |
Http://www.rhyme.com.au | / \|
| --________--
PGP key available upon request, | /
and from pgp5.ai.mit.edu:11371 |/
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2000-08-07 16:53:50 | Re: [HACKERS] Re: Trouble with float4 afterupgrading from 6.5.3 to 7.0.2 |
Previous Message | Philip Warner | 2000-08-07 16:15:45 | Re: [HACKERS] Re: Trouble with float4 after upgrading from 6.5.3 to 7.0.2 |
From | Date | Subject | |
---|---|---|---|
Next Message | Thomas Lockhart | 2000-08-07 16:30:30 | Re: mac.c |
Previous Message | Zeugswetter Andreas SB | 2000-08-07 16:15:59 | AW: Anyone particularly wedded to func_tlist mechanism? |