Re: WIP: rewrite numeric division

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:

> Gregory Stark <stark(at)enterprisedb(dot)com> writes:
>> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>>> Yeah. I was basically waiting to see if anyone could come up with a
>>> faster solution. Since no one seems to have an idea how to do it
>>> better, I'm inclined to apply the patch for 8.3.
>
>> My only reservation is that I don't have the numeric methods background to
>> tell if it's really necessary. My fix does resolve the only actual documented
>> inaccuracy in the existing method.
>
> Well, this doesn't take a lot of numerical methods background: the
> fundamental problem is that the existing code generates an *approximate*
> answer, whereas people who are doing div and mod on large integers tend
> to expect an *exact* answer. Approximate doesn't cut it --- there will
> always be cases where an off-by-one-in-the-last-internal-place error can
> carry far enough to the left to be visible to the user.

So does your code behave differently for this or does it round off to the
arbitrary division precision before hitting trunc?

postgres=# select trunc(99999999999999999999::numeric / 1000000000::numeric);
trunc
--------------
100000000000
(1 row)

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com