Re: rounding problems

From: Justin <justin(at)emproshunts(dot)com>
To: pgsql-general(at)postgresql(dot)org
Subject: Re: rounding problems
Date: 2008-05-13 17:29:07
Message-ID: 4829CFE3.5040803@emproshunts.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general


Tomasz Ostrowski wrote:
> On 2008-05-13 17:29, Justin wrote:
>
>> Tomasz Ostrowski wrote:
>>
>>> On 2008-05-12 20:49, Justin wrote:
>>>
>>>
>>>
>>>> We take (List Price * discount Percent) * Number of Pieces = net
>>>> price.
>>>>
>>>>
>>> This is wrong. You should do in Excel:
>>> ( price * amount ) * discount
>>> As otherwise any small error in representation of price*discount would
>>> be multiplied by usually high amount.
>>>
>>>
>> Your saying in Excel, Multiplication is not Commutativity??? that
>> sends shudders down my back
>>
>
> On floats it is not in any language. Try this C program:
>
> #include <stdio.h>
>
> int main()
> {
> float a = 0.1;
> float b = 10;
> float c = 1000000000;
> float d = a * b;
> printf("%.10f\n", d*c);
> d = b * c;
> printf("%.10f\n", a*d);
> return 0;
> }
>
> On my Intel 32-bit I get:
> 1000000000.0000000000
> 1000000014.9011611938
>
I ran this on Windows using MSVC

The Result are the same when using float . But the Ms compiler throws
warnings, crying truncation and rounding problems
1000000000.0000000000
1000000014.9011612000

I change it to double problem goes away.
1000000000.0000000000
1000000000.0000000000

I have always avoided floating points. In all the work i have done thus
far the problems with floating math would cause scrap parts or make
Quality control worthless.

Accuracy trumps pretty much every concern around here.

> Pozdrawiam
> Tometzky
>

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Robert Treat 2008-05-13 18:13:00 Re: PG -v- MySQL
Previous Message Justin 2008-05-13 17:01:11 Re: rounding problems