Our FLOAT(p) precision does not conform to spec

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org, pgsql-sql(at)postgreSQL(dot)org
Subject: Our FLOAT(p) precision does not conform to spec
Date: 2003-06-16 22:15:46
Message-ID: 7114.1055801746@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-sql

Fernando Nasser of Red Hat pointed out to me that we are not quite
spec-compliant on the FLOAT(p) datatype notation. We interpret P as
the number of decimal digits of precision, and hence translate

P = 1..6 => float4 (a/k/a REAL)
P = 7..15 => float8 (a/k/a DOUBLE PRECISION)
otherwise error

However, the spec is perfectly clear that P is to be interpreted as
the number of *binary* digits of precision, not decimal digits. SQL92
section 4.4.1 says:

An approximate numeric value consists of a mantissa and an expo-
nent. The mantissa is a signed numeric value, and the exponent is
a signed integer that specifies the magnitude of the mantissa. An
approximate numeric value has a precision. The precision is a posi-
tive integer that specifies the number of significant binary digits
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
in the mantissa. The value of an approximate numeric value is the
mantissa multiplied by 10^exponent.

So it's fairly clear that P is not the number of decimal digits. (The
reference to multiplying by 10^exponent seems bogus, since on machines
where the mantissa is in fact binary, one would expect a base-2 or
possibly base-16 exponent to be used. But this does not affect the
precision of the mantissa AFAICS.)

On the assumption that most platforms have IEEE float math, it would be
appropriate to interpret P like this:

P = 1..24 => float4
P = 25..53 => float8
otherwise error

This is a straightforward change and would not break pg_dump files,
since fortunately pg_dump always references the underlying types and
never refers to anything as FLOAT(p). But I wonder whether it is
likely to break many existing applications. There is a hazard of some
existing app asking for (what it thinks is) float8 and getting float4
instead.

Is it worth trying to provide some sort of backwards-compatibility mode?
We could imagine adding a GUC variable to select binary or decimal
precision, but I really don't want to. It would increase the amount of
work needed by more than an order of magnitude, and this problem doesn't
seem worth it. I'd rather just list this under Incompatibilities in the
7.4 release notes.

Comments?

regards, tom lane

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2003-06-16 23:35:59 Re: [HACKERS] Our FLOAT(p) precision does not conform to spec
Previous Message Bruce Momjian 2003-06-16 21:21:01 Re: Errors compiling hba.c in current CVS

Browse pgsql-sql by date

  From Date Subject
Next Message weigelt 2003-06-16 22:20:39 unique keys on inherited classes
Previous Message Bu, Ning 2003-06-16 22:02:34 set decimal point in SQL