Re: numeric/decimal docs bug?

From: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
To: pgman(at)candle(dot)pha(dot)pa(dot)us
Cc: janwieck(at)yahoo(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, peter_e(at)gmx(dot)net, pgsql-hackers(at)postgresql(dot)org
Subject: Re: numeric/decimal docs bug?
Date: 2002-03-12 02:01:17
Message-ID: 20020312110117E.t-ishii@sra.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> Jan Wieck wrote:
> > > The hard limit is certainly no more than 64K, since we store these
> > > numbers in half of an atttypmod. In practice I suspect the limit may
> > > be less; Jan would be more likely to remember...
> >
> > It is arbitrary of course. I don't recall completely, have to
> > dig into the code, but there might be some side effect when
> > mucking with it.
> >
> > The NUMERIC code increases the actual internal precision when
> > doing multiply and divide, what happens a gazillion times
> > when doing higher functions like trigonometry. I think there
> > was some connection between the max precision and how high
> > this internal precision can grow, so increasing the precision
> > might affect the computational performance of such higher
> > functions significantly.
>
> Oh, interesting, maybe we should just leave it alone.

So are we going to just fix the docs?
--
Tatsuo Ishii

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Justin Clift 2002-03-12 02:10:42 Re: Promising results with Intel Linux x86 compiler
Previous Message Dave Cramer 2002-03-12 01:44:03 Re: Object?? -Relational DBMS Postgresql are you sure?