Re: numeric/decimal docs bug?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Jan Wieck <janwieck(at)yahoo(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Peter Eisentraut <peter_e(at)gmx(dot)net>, Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>, pgsql-hackers(at)postgresql(dot)org, Jan Wieck <JanWieck(at)yahoo(dot)com>
Subject: Re: numeric/decimal docs bug?
Date: 2002-03-11 22:02:25
Message-ID: 200203112202.g2BM2Pp19522@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Jan Wieck wrote:
> > The hard limit is certainly no more than 64K, since we store these
> > numbers in half of an atttypmod. In practice I suspect the limit may
> > be less; Jan would be more likely to remember...
>
> It is arbitrary of course. I don't recall completely, have to
> dig into the code, but there might be some side effect when
> mucking with it.
>
> The NUMERIC code increases the actual internal precision when
> doing multiply and divide, what happens a gazillion times
> when doing higher functions like trigonometry. I think there
> was some connection between the max precision and how high
> this internal precision can grow, so increasing the precision
> might affect the computational performance of such higher
> functions significantly.

Oh, interesting, maybe we should just leave it alone.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2002-03-11 22:11:24 Re: INDEX_MAX_KEYS
Previous Message Jan Wieck 2002-03-11 21:50:09 Re: INDEX_MAX_KEYS