Re: numeric/decimal docs bug?

From: Jan Wieck <janwieck(at)yahoo(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>, pgsql-hackers(at)postgresql(dot)org, Jan Wieck <JanWieck(at)yahoo(dot)com>
Subject: Re: numeric/decimal docs bug?
Date: 2002-03-11 21:04:44
Message-ID: 200203112104.g2BL4i828375@saturn.janwieck.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tom Lane wrote:
> Peter Eisentraut <peter_e(at)gmx(dot)net> writes:
> > Tom Lane writes:
> > #define NUMERIC_MAX_PRECISION 1000
> >>
> >> I was thinking just the other day that there's no reason for that
> >> limit to be so low. Jan, couldn't we bump it up to 8 or 16K or so?
>
> > Why have an arbitrary limit at all? Set it to INT_MAX,
>
> The hard limit is certainly no more than 64K, since we store these
> numbers in half of an atttypmod. In practice I suspect the limit may
> be less; Jan would be more likely to remember...

It is arbitrary of course. I don't recall completely, have to
dig into the code, but there might be some side effect when
mucking with it.

The NUMERIC code increases the actual internal precision when
doing multiply and divide, what happens a gazillion times
when doing higher functions like trigonometry. I think there
was some connection between the max precision and how high
this internal precision can grow, so increasing the precision
might affect the computational performance of such higher
functions significantly.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #

_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2002-03-11 21:21:17 Re: Do FROM items of different schemas conflict?
Previous Message Mark Pritchard 2002-03-11 20:56:23 Re: Do FROM items of different schemas conflict?