Re: numeric/decimal docs bug?

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Tatsuo Ishii <t-ishii(at)sra(dot)co(dot)jp>
Cc: janwieck(at)yahoo(dot)com, tgl(at)sss(dot)pgh(dot)pa(dot)us, peter_e(at)gmx(dot)net, pgsql-hackers(at)postgresql(dot)org
Subject: Re: numeric/decimal docs bug?
Date: 2002-04-13 01:37:34
Message-ID: 200204130137.g3D1bY213251@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tatsuo Ishii wrote:
> > Jan Wieck wrote:
> > > > The hard limit is certainly no more than 64K, since we store these
> > > > numbers in half of an atttypmod. In practice I suspect the limit may
> > > > be less; Jan would be more likely to remember...
> > >
> > > It is arbitrary of course. I don't recall completely, have to
> > > dig into the code, but there might be some side effect when
> > > mucking with it.
> > >
> > > The NUMERIC code increases the actual internal precision when
> > > doing multiply and divide, what happens a gazillion times
> > > when doing higher functions like trigonometry. I think there
> > > was some connection between the max precision and how high
> > > this internal precision can grow, so increasing the precision
> > > might affect the computational performance of such higher
> > > functions significantly.
> >
> > Oh, interesting, maybe we should just leave it alone.
>
> So are we going to just fix the docs?

OK, I have updated the docs. Patch attached.

I have also added this to the TODO list:

* Change NUMERIC to enforce the maximum precision, and increase it

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 853-3000
+ If your life is a hard drive, | 830 Blythe Avenue
+ Christ can be your backup. | Drexel Hill, Pennsylvania 19026

Attachment Content-Type Size
unknown_filename text/plain 1.3 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2002-04-13 01:39:09 Re: numeric/decimal docs bug?
Previous Message Neil Conway 2002-04-13 01:28:48 Re: pg_dump is broken in CVS tip