Re: float output precision questions

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Pedro M(dot) Ferreira" <pfrazao(at)ualg(dot)pt>
Cc: Stephan Szabo <sszabo(at)megazone23(dot)bigpanda(dot)com>, Peter Eisentraut <peter_e(at)gmx(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: float output precision questions
Date: 2002-10-31 17:58:21
Message-ID: 8166.1036087101@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

"Pedro M. Ferreira" <pfrazao(at)ualg(dot)pt> writes:
> Have two parameters, say DOUBLE_OUTPUT and EXTRA_DIGITS. DOUBLE_OUTPUT
> would select from decimal output or normalized output. EXTRA_DIGITS
> would add the required extra digits, from 0 (default) to 3, when output
> is decimal.

I'm not happy with adding the hex-output option, since it's not
very portable and doesn't seem necessary to solve the problem anyway.

But I think an EXTRA_DIGITS setting might be interesting. In
particular, suppose we allowed EXTRA_DIGITS to be negative? Setting
it to -1 or -2 would go a long way towards eliminating our problems
with platform variations in the geometry regression test.

Perhaps something like

extra_float_digits int range -2 to 2, default 0

extra_float_digits adjusts the number of digits displayed for float4 and
float8 output; the base value of 0 means we output FLT_DIG or DBL_DIG
digits respectively.

Per discussion, there's no reason to allow a value greater than 2, but
I'm not as sure what the lower limit should be --- maybe there's some
use in setting it less than -2?

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pedro M. Ferreira 2002-10-31 17:58:51 Re: float output precision questions
Previous Message Rod Taylor 2002-10-31 17:56:24 Re: 7.2.3 vacuum bug