Re: Rounding problems

From: Justin <justin(at)emproshunts(dot)com>
To: Paolo Saudin <paolo(at)ecometer(dot)it>, "pgsql-general(at)postgresql(dot)org" <pgsql-general(at)postgresql(dot)org>
Subject: Re: Rounding problems
Date: 2009-05-03 14:42:13
Message-ID: 49FDAD45.4070104@emproshunts.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
<br>
<br>
Paolo Saudin wrote:
<blockquote cite="mid:002801c9cbeb$f6b59c20$e420d460$(at)it" type="cite">
<meta http-equiv="Content-Type" content="text/html; ">
<meta name="Generator" content="Microsoft Word 12 (filtered medium)">
<style>
<!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
span.StileMessaggioDiPostaElettronica17
{mso-style-type:personal-compose;
font-family:"Calibri","sans-serif";
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;}
@page Section1
{size:612.0pt 792.0pt;
margin:70.85pt 2.0cm 2.0cm 2.0cm;}
div.Section1
{page:Section1;}
/* List Definitions */
@list l0
{mso-list-id:361589963;
mso-list-type:hybrid;
mso-list-template-ids:-1476211862 -1560001026 68157443 68157445 68157441 68157443 68157445 68157441 68157443 68157445;}
@list l0:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:\F06E;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l1
{mso-list-id:892347903;
mso-list-type:hybrid;
mso-list-template-ids:686571502 512508354 68157443 68157445 68157441 68157443 68157445 68157441 68157443 68157445;}
@list l1:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:\F06E;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
@list l2
{mso-list-id:989483843;
mso-list-type:hybrid;
mso-list-template-ids:-722197268 311070654 68157443 68157445 68157441 68157443 68157445 68157441 68157443 68157445;}
@list l2:level1
{mso-level-start-at:0;
mso-level-number-format:bullet;
mso-level-text:\F06E;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;
mso-fareast-font-family:Calibri;
mso-bidi-font-family:"Times New Roman";}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style>
<!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
<div class="Section1">
<p class="MsoNormal">Hi,<o:p></o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">I have a problem with a query wich simple
aggregate values.
In the sample below I have two values, 1.3 and 1.4. Rounding their
average with
one decimals, should give 1.4.<o:p></o:p></p>
<p class="MsoNormal">The first query with &nbsp;- &nbsp;cast(
tables_seb.tbl_arvier_chamencon.id_1&nbsp; AS numeric) AS value &nbsp;- give
the expected result, while the second one with -
tables_seb.tbl_arvier_chamencon.id_1
AS value - give 1.3. <o:p></o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<p class="MsoNormal">Which could be the reason ??<o:p></o:p></p>
<p class="MsoNormal"><o:p>&nbsp;</o:p></p>
<br>
</div>
</blockquote>
My first thought is whats with all the castings???&nbsp;&nbsp; <br>
<br>
Castings&nbsp; are mostly likely the cause of your problems,<b> &nbsp; </b>What
is <b>tbl_arvier_chamencon.id_1 </b>data type???<br>
<br>
I'm guessing its something other than numeric. All other floating point
data types will have problems caused by Binary Floating-Point Arithmetic<br>
<br>
Numeric data type uses different functions to do its math for the
stated purpose of being exact yet being allot slower. <br>
<br>
In one query casting is done prior to avg() yet in the other casting is
done after avg().&nbsp;&nbsp; This will allow Postgres to use different functions
to calculate average giving an unexpected result.<br>
</body>
</html>

Attachment Content-Type Size
unknown_filename text/html 4.6 KB

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Tom Lane 2009-05-03 14:52:45 Re: Rounding problems
Previous Message Craig Ringer 2009-05-03 13:45:12 Re: Server timestamp