Skip site navigation (1) Skip section navigation (2)

Re: Numeric 508 datatype

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
Cc: Simon Riggs <simon(at)2ndquadrant(dot)com>, pgsql-patches(at)postgresql(dot)org
Subject: Re: Numeric 508 datatype
Date: 2005-12-02 17:17:14
Message-ID: 200512021717.jB2HHEh15358@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-generalpgsql-hackerspgsql-patches
Bruce Momjian wrote:
> OK, seems all objections have been dealt with so it goes into the patch
> queue.  I will ask on 'general'.
> 
> The only downside I see is that I can't impress people by doing:
> 
> 	SELECT factorial(4000);
> 
> I don't suppose the _impression_ factor is worth two bytes per value. 
> Shame.
> 
> I suppose people wanting to do such manipulations will have to store the
> numbers as text and use a server-side library like perl to do
> calculations.

Oops, I was wrong about this.  The patch changes the maximum _specified_
precision:
	
	  /*
	!  * Hardcoded precision limit - arbitrary, but must be small enough that
	!  * dscale values will fit in 14 bits.
	   */
	! #define NUMERIC_MAX_PRECISION     1000
	
	  /*
	   * Internal limits on the scales chosen for calculation results
	--- 15,23 ----
	  #define _PG_NUMERIC_H_
	
	  /*
	!  * Hardcoded precision limit - maximum that can fit in Numeric storage
	   */
	! #define NUMERIC_MAX_PRECISION     508

but in fact, our computational precision is 4096, and we silently
overflow for values greater than that:

	test=> create table test(x numeric);
	CREATE TABLE
	test=> insert into test values (factorial(4000));
	INSERT 0 1

The length is 4096 digits, and so is factorial(10000) --- clearly wrong.
I now see in the TODO:

	* Change NUMERIC to enforce the maximum precision, and increase it

So we are really decreasing the specified precision from 1000 to 508,
and the computational precision from 4096 to 508.  Is there any plan to
fix the silent overflow problem?  Is that in the patch?  I don't see it.

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
  +  If your life is a hard drive,     |  13 Roberts Road
  +  Christ can be your backup.        |  Newtown Square, Pennsylvania 19073

In response to

Responses

pgsql-hackers by date

Next:From: Jaime CasanovaDate: 2005-12-02 17:25:58
Subject: Re: 8.1, OID's and plpgsql
Previous:From: Chris BrowneDate: 2005-12-02 17:14:09
Subject: Re: [HACKERS] Should libedit be preferred to libreadline?

pgsql-patches by date

Next:From: Bruce MomjianDate: 2005-12-02 17:26:53
Subject: Re: Case Conversion Fix for MB Chars
Previous:From: Chris BrowneDate: 2005-12-02 17:14:09
Subject: Re: [HACKERS] Should libedit be preferred to libreadline?

pgsql-general by date

Next:From: Jaime CasanovaDate: 2005-12-02 17:25:58
Subject: Re: 8.1, OID's and plpgsql
Previous:From: Eric EDate: 2005-12-02 17:16:18
Subject: Re: How: single db, multiple users

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group