Re: Re: [Oledb-dev] double precision error with pg linux server, but not with windows pg server

From: Shachar Shemesh <shachar(at)shemesh(dot)biz>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, pgsql-hackers(at)postgresql(dot)org, Greg Smith <gsmith(at)gregsmith(dot)com>, oledb-devel(at)pgfoundry(dot)org
Subject: Re: Re: [Oledb-dev] double precision error with pg linux server, but not with windows pg server
Date: 2007-05-24 05:24:58
Message-ID: 465521AA.4010505@shemesh.biz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Shachar Shemesh <shachar(at)shemesh(dot)biz> writes:
>
>> Before you explode at me (again :), I'm not arguing that you can do
>> binary based calculations of decimal numbers without having rounding
>> errors that come to bite you. I know you can't. What I'm saying is that
>> we have two cases to consider. In one of them the above is irrelevant,
>> and in the other I'm not so sure it's true.
>>
>
> You're setting up a straw-man argument, though.
I was answering your claim that it's impossible to convert decimal to
binary based floats without rounding errors.
> The real-world problem
> cases here are not decimal, they are non-IEEE binary floating
> arithmetic. The typical difference from IEEE is slightly different
> tradeoffs in number of mantissa bits vs number of exponent bits within a
> 32- or 64-bit value.
I answered that elsewhere while suggesting a different format that would
address that. These numbers do not appear to be a concern in our
situation, however.
> I seem to recall also that there are machines that
> treat the exponent as power-of-16 not power-of-2.
I'm pretty sure I don't understand this. Maybe I misunderstood the
format, but wouldn't that actually lose you precision with, at most,
marginal gain in range? As far as I can see, the moment you no longer
work in base 2 you lose the implicit bit, which means you have a one bit
less starting point than base 2 notations (all number are denormalized).
> So depending on which
> way the tradeoffs went, the other format will have either more precision
> or more range than IEEE.
>
Again, should that be a real concern, see my message at
http://archives.postgresql.org/pgsql-hackers/2007-05/msg00892.php for
details about what the suggestion actually is. Just be sure to read
"IEEE" there as meaning "IEEE like". I allowed different sizes for the
fields.
> regards, tom lane
>
Shachar

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2007-05-24 06:04:12 Re: Re: [Oledb-dev] double precision error with pg linux server, but not with windows pg server
Previous Message Tom Lane 2007-05-24 05:16:34 Re: Concurrent psql patch