Re: Numeric 508 datatype

Bruce Momjian wrote:
> OK, seems all objections have been dealt with so it goes into the patch
> queue. I will ask on 'general'.
>
> The only downside I see is that I can't impress people by doing:
>
> SELECT factorial(4000);
>
> I don't suppose the _impression_ factor is worth two bytes per value.
> Shame.
>
> I suppose people wanting to do such manipulations will have to store the
> numbers as text and use a server-side library like perl to do
> calculations.

Oops, I was wrong about this. The patch changes the maximum _specified_
precision:

/*
! * Hardcoded precision limit - arbitrary, but must be small enough that
! * dscale values will fit in 14 bits.
*/
! #define NUMERIC_MAX_PRECISION 1000

/*
* Internal limits on the scales chosen for calculation results
--- 15,23 ----
#define _PG_NUMERIC_H_

/*
! * Hardcoded precision limit - maximum that can fit in Numeric storage
*/
! #define NUMERIC_MAX_PRECISION 508

but in fact, our computational precision is 4096, and we silently
overflow for values greater than that:

test=> create table test(x numeric);
CREATE TABLE
test=> insert into test values (factorial(4000));
INSERT 0 1

The length is 4096 digits, and so is factorial(10000) --- clearly wrong.
I now see in the TODO:

* Change NUMERIC to enforce the maximum precision, and increase it

So we are really decreasing the specified precision from 1000 to 508,
and the computational precision from 4096 to 508. Is there any plan to
fix the silent overflow problem? Is that in the patch? I don't see it.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073