Re: WIP: rewrite numeric division

Gregory Stark <stark(at)enterprisedb(dot)com> writes:
> "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> writes:
>> Well, this doesn't take a lot of numerical methods background: the
>> fundamental problem is that the existing code generates an *approximate*
>> answer, whereas people who are doing div and mod on large integers tend
>> to expect an *exact* answer. Approximate doesn't cut it --- there will
>> always be cases where an off-by-one-in-the-last-internal-place error can
>> carry far enough to the left to be visible to the user.

> So does your code behave differently for this or does it round off to the
> arbitrary division precision before hitting trunc?

> postgres=# select trunc(99999999999999999999::numeric / 1000000000::numeric);
> trunc
> --------------
> 100000000000
> (1 row)

No, my proposed patch doesn't change that. It might be that we should
provide an "integer division" operator for NUMERIC, so that you can get
at the exact result of trunc(x/y).

regards, tom lane