Re: [HACKERS] 123.45 - 123 = 0.45

From: Bruce Momjian <maillist(at)candle(dot)pha(dot)pa(dot)us>
To: djackson(at)cpsgroup(dot)com (Jackson, DeJuan)
Cc: daveh(at)insightdist(dot)com, sferac(at)bo(dot)nettuno(dot)it, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: [HACKERS] 123.45 - 123 = 0.45
Date: 1998-12-15 16:53:26
Message-ID: 199812151653.LAA16720@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> > Yes, someone pointed this out to me in private e-mail, and I wrote a C
> > program to confirm it. I just had never seen such rounding on such
> > small non-irrational numbers.
> But, the problem is that it is irrational in base 2 (or at least so many
> digits that it won't fit).

Ah, I figured that out after I sent the message.

--
Bruce Momjian | http://www.op.net/~candle
maillist(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas G. Lockhart 1998-12-15 19:55:00 Re: [COMMITTERS] 'pgsql/src/backend/utils/adt nabstime.c'
Previous Message Tom Lane 1998-12-15 16:43:08 Re: [HACKERS] 123.45 - 123 = 0.45