Skip site navigation (1) Skip section navigation (2)

Re: Arbitrary precision modulo operation

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Dann Corbit" <DCorbit(at)connx(dot)com>
Cc: "Bruno Wolff III" <bruno(at)wolff(dot)to>,"Paul Tillotson" <pntil(at)shentel(dot)net>, pgsql-general(at)postgresql(dot)org
Subject: Re: Arbitrary precision modulo operation
Date: 2004-04-29 21:31:28
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-general
"Dann Corbit" <DCorbit(at)connx(dot)com> writes:
> From: Tom Lane [mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us] 
>> Fine.  How many is that, exactly?

> Here is what I would suggest:
> Using the outline I proposed before (starting with a floating point
> divide of DBL_DIG digits of precision), keep doubling the precision
> until the precision is 5 digits larger than either operand.  If the last
> doubling makes the precision larger (quite likely) simply reduce it to
> the smaller margin.

And this guarantees a correct answer why?

AFAIK div_var is already correct per its spec, which is that it
generates an answer rounded to the requested number of digits.
The question at hand is what number of digits to request.

After thinking about it I don't see any reason that DBL_DIG has
anything to do with a non-surprising answer ... much less "DBL_DIG + 5"
which seems picked out of the air ...

			regards, tom lane

In response to

pgsql-general by date

Next:From: Dann CorbitDate: 2004-04-29 21:35:46
Subject: Re: Arbitrary precision modulo operation
Previous:From: Jean-Luc LachanceDate: 2004-04-29 21:06:00
Subject: Re: Timestamp problems...wrong weeks.

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group