Re: Arbitrary precision modulo operation

Paul Tillotson <pntil(at)shentel(dot)net> writes:
> mod(x, y) is computed as x - trunc(x / y) * y in the mod_var() function

> However, it appears that the division operator itself is rounding up,

This is because div_var rounds its output to the number of fractional
digits specified by select_div_scale, which saith

/*
* The result scale of a division isn't specified in any SQL standard.
* For PostgreSQL we select a result scale that will give at least
* NUMERIC_MIN_SIG_DIGITS significant digits, so that numeric gives a
* result no less accurate than float8; but use a scale not less than
* either input's display scale.
*/

You get the "correct" answer if you force a couple more digits to be
carried by increasing either input's dscale, eg

regression=# select 123456789012345678901234567890.00 % 123;
?column?
----------
117.00
(1 row)

It could be that select_div_scale needs to allow some more slop, or that
that routine is okay but mod_var shouldn't depend on it to select the
intermediate result scale for its internal division. Comments?

regards, tom lane