Question about the TODO, numerics, and division

From: Chris Travers <chris(at)metatrontech(dot)com>
To: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Question about the TODO, numerics, and division
Date: 2007-03-20 22:11:43
Message-ID: 46005C1F.5040607@metatrontech.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi all;

I have been looking at the TODO and have found something that I find
sort of odd and we should probably reconsider:

One of the items under data types is:

* Add NUMERIC division operator that doesn't round?

Currently NUMERIC _rounds_ the result to the specified precision.
This means division can return a result that multiplied by the
divisor is greater than the dividend, e.g. this returns a value > 10:

SELECT (10::numeric(2,0) / 6::numeric(2,0))::numeric(2,0) * 6;

This does not seem to me to be an division op issue but rather a simple
casting mistake. Note that the result of 10/6 is cast as numeric(2,0)
and then multiplied by 6.

The following example shows that the problem is with the query and
casting, not with the division op:
SELECT ((10::numeric(2,0) / 6::numeric(2,0)) * 6)::numeric(2,0);
numeric
---------
10
(1 row)

Am I missing something?

Best Wishes,
Chris Travers

Attachment Content-Type Size
chris.vcf text/x-vcard 171 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Gavin Sherry 2007-03-20 22:36:55 Re: Bitmapscan changes - Requesting further feedback
Previous Message Neil Conway 2007-03-20 21:19:05 Re: patch adding new regexp functions