Re: Question about the TODO, numerics, and division

From: Martijn van Oosterhout <kleptog(at)svana(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Chris Travers <chris(at)verkiel(dot)metatrontech(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Question about the TODO, numerics, and division
Date: 2007-03-21 12:26:55
Message-ID: 20070321122655.GA13787@svana.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Tue, Mar 20, 2007 at 08:27:46PM -0400, Tom Lane wrote:
> and of course if you multiply that by 6 you get
>
> regression=# SELECT (10::numeric(2,0) / 6::numeric(2,0)) * 6;
> ?column?
> ---------------------
> 10.0000000000000002
> (1 row)

The only way to fix that is to shift from fixed-point arithmetic to
something that manipulates rationals, where 10/6 can be represented
exactly.

Arithmatic on rationals is not terribly complicated, the real problem
comes with the fact that as the number of operations increase, your
numerator and denominator are going to tend to infinity in an attempt
to represent your number accuratly. That's probably a solved problem
too, but still...

Have a nice day,
--
Martijn van Oosterhout <kleptog(at)svana(dot)org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Gregory Stark 2007-03-21 12:34:12 Re: patch adding new regexp functions
Previous Message Gregory Stark 2007-03-21 12:21:07 Re: Question: pg_class attributes and race conditions ?