Re: [HACKERS] 123.45 - 123 = 0.45

From: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>
To: daveh(at)insightdist(dot)com (David Hartwig)
Cc: sferac(at)bo(dot)nettuno(dot)it, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: [HACKERS] 123.45 - 123 = 0.45
Date: 1998-12-15 16:20:15
Message-ID: 199812151620.LAA16225@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> Try this:
>
> #include <stdio.h>
>
> main()
> {
> double f1 = 123.45;
> double f2 = 123.00;
> double r;
>
> r = f1 - f2;
> printf("%0.15f %0.15f %0.15f\n", f1, f2, r);
> }
>
> Internal representation of 123.45 is not exact. In the conversion to
> binary, an irrational number is created which is truncated to 64 bits.

Yes, someone pointed this out to me in private e-mail, and I wrote a C
program to confirm it. I just had never seen such rounding on such
small non-irrational numbers.

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Jackson, DeJuan 1998-12-15 16:31:29 RE: [HACKERS] 123.45 - 123 = 0.45
Previous Message Sferacarta Software 1998-12-15 15:35:04 Re[2]: [HACKERS] 123.45 - 123 = 0.45