Re: BUG #2037: user function call unexpected "input out of range"

From: tomas(at)tuxteam(dot)de (Tomas Zerolo)
To: Tom <lackey(at)ltu(dot)edu>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: BUG #2037: user function call unexpected "input out of range"
Date: 2005-11-12 17:08:17
Message-ID: 20051112170817.GA11828@www.trapp.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

On Fri, Nov 11, 2005 at 09:26:47PM +0000, Tom wrote:
>
> The following bug has been logged online:
>
> Bug reference: 2037
> Logged by: Tom
> Email address: lackey(at)ltu(dot)edu
> PostgreSQL version: 8.1-beta4
> Operating system: Windows XP
> Description: user function call unexpected "input out of range"
> Details:

[ basically acos(sin(w1)*sin(w2) + cos(w1)*cos(w2)*cos(l1-l2)) ]

Seems to me that you are hitting an unfortunate roundoff error, where
the argument to acos is slightly greater than 1 (it will be
mathematically 1 in the case w1==w2 and l1==l2).

Maybe you should limit the argument to acos to -1..+1?

regards
-- tomás

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message Tom Lane 2005-11-12 17:18:49 Re: BUG #2037: user function call unexpected "input out of range"
Previous Message Tom Lane 2005-11-12 15:50:42 Re: storage sync failed on magnetic disk: Permission denied