Re: float output precision questions

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Pedro M(dot) Ferreira" <pfrazao(at)ualg(dot)pt>
Cc: Stephan Szabo <sszabo(at)megazone23(dot)bigpanda(dot)com>, Peter Eisentraut <peter_e(at)gmx(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: float output precision questions
Date: 2002-10-31 16:01:21
Message-ID: 7322.1036080081@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

"Pedro M. Ferreira" <pfrazao(at)ualg(dot)pt> writes:
> Its like I said before, the guys from matlab (in x86 IEEE float) go to
> DBL_BIG+3 to have 'maximum precision'.

Apparently they have not read the canonical papers in the field.
[ googles for a moment... ] See

How to read floating point numbers accurately
William D. Clinger

How to print floating-point numbers accurately
Guy L. Steele, Jr., Jon L. White

both published at the 1990 ACM Conference on Programming Language Design
and Implementation and subsequently reprinted in ACM SIGPLAN Notices
Volume 25, Issue 6 (June 1990).

I was misremembering these papers to claim DBL_DIG+1 is enough, but
actually they prove that DBL_DIG+2 is necessary and sufficient (and
give code to do it correctly, too).

Printing DBL_DIG+3 is just producing an extra garbage digit; it won't
help matters. Any reasonably well-written C library is going to be
able to reproduce a double value with DBL_DIG+2 digits of I/O; and if
it's not well-written, I would have no confidence in its ability to do
so with DBL_DIG+3 digits...

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Rod Taylor 2002-10-31 16:15:31 Re: setuid for defaults, constraints and triggers (Was:
Previous Message Tom Lane 2002-10-31 15:40:21 Re: float output precision questions