Re: Add numeric_trim(numeric)

Hi

2015-11-19 3:58 GMT+01:00 Marko Tiikkaja <marko(at)joh(dot)to>:

> Hi,
>
> Here's a patch for the second function suggested in
> 5643125E(dot)1030605(at)joh(dot)to(dot) This is my first patch trying to do anything
> with numerics, so please be gentle. I'm sure it's full of stupid.
>
> January's commit fest, feedback welcome, yada yada..
>

I am looking on this patch and I don't understand to formula

dscale = (ndigits - arg.weight - 1) * DEC_DIGITS;

the following rule is valid

DEC_DIGITS * ndigits >= dscale + arg.weight + 1

so dscale should be calculated like

dscale <= DEC_DIGITS * ndigits - arg.weight - 1

?

but your formula is correct and working. Can you explain it?

Regards

Pavel

>
>
> .m
>