Skip site navigation (1) Skip section navigation (2)

Re: Function trunc() behaves in unexpected manner with different data types

From: Greg Stark <gsstark(at)mit(dot)edu>
To: Merlin Moncure <mmoncure(at)gmail(dot)com>
Cc: "Nathan M(dot) Davalos" <n(dot)davalos(at)sharedmarketing(dot)com>, pgsql-bugs(at)postgresql(dot)org
Subject: Re: Function trunc() behaves in unexpected manner with different data types
Date: 2011-02-25 02:03:40
Message-ID: AANLkTin81WeP3+gOjyp6bX-m26vUjcBRBd2GRGkWwMxZ@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-bugs
On Thu, Feb 24, 2011 at 7:31 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> the root issue I think here is that the string version of the double
> precision math is approximated:

No, it's simpler than that, all double precision math is approximated.
The root issue is that 2183.67 is not representable in a floating
point binary number. Just like 1/3 isn't representable in base 10
(decimal) numbers many fractions aren't representable in base 2
(binary) numbers. The result are repeated decimals like 0.3333... if you
multiply that by three you get 0.99999 and if you truncate that you
get 0 insted of 1.

It's the trunc() that's exposing the imprecision because like "=" it
depends on the precise value of the number down to the last digit.
Though depending on the arithmetic you can always make the precision
expand beyond the last digit anyways -- when you multiply by 100 you
magnify that imprecision too.

-- 
greg

In response to

Responses

pgsql-bugs by date

Next:From: Jakub OuhrabkaDate: 2011-02-25 11:20:33
Subject: Corrupted index on 9.0.3 streaming hot standby
Previous:From: Tom LaneDate: 2011-02-24 23:23:47
Subject: Re: LOCALTIMESTAMP has wrong time zone

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group