Skip site navigation (1) Skip section navigation (2)

Re: my float function returns NaN

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Nathaniel Trellice <naptrel(at)yahoo(dot)co(dot)uk>
Cc: Frank Bax <fbax(at)sympatico(dot)ca>, "pgsql-novice(at)postgresql(dot)org" <pgsql-novice(at)postgresql(dot)org>
Subject: Re: my float function returns NaN
Date: 2012-04-07 14:54:22
Message-ID: 2542.1333810462@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-novice
Nathaniel Trellice <naptrel(at)yahoo(dot)co(dot)uk> writes:
> My guess would be that due to rounding errors, the argument passed to
> acos() is outwith the interval [-1,1] and acos() is only defined
> within this interval (for reals).

Yeah.  It's not helping any that you've defined geo_deg2rad() to do its
arithmetic in float (that is float4) precision.  Making it work in
float8 would probably improve matters.  Still, I agree that switching to
a more numerically stable distance calculation would be a good idea.

			regards, tom lane

In response to

Responses

pgsql-novice by date

Next:From: Frank BaxDate: 2012-04-07 16:24:56
Subject: Re: my float function returns NaN
Previous:From: Nathaniel TrelliceDate: 2012-04-07 08:32:35
Subject: Re: my float function returns NaN

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group