Skip site navigation (1) Skip section navigation (2)

Re: modulo operator bug?

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Kemal Bayram <kemal(at)corenetworks(dot)co(dot)uk>
Cc: pgsql-bugs(at)postgresql(dot)org
Subject: Re: modulo operator bug?
Date: 2001-03-14 16:56:44
Message-ID: 27075.984589004@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-bugs
> Kemal Bayram <kemal(at)corenetworks(dot)co(dot)uk> writes:
>> from psql:
>> kemal=# select 0.99 % 0.50;
>> ?column? 
>> ----------
>> 0.04
>> (1 row)

>> Am I missing something fundemental here or is this indeed a bug?

> Looks like a bug to me too.  mod_var shouldn't be using tmp.rscale,
> I think.

OK, fixed for 7.1.  If you need a fix in 7.0.*, the attached patch should
apply to 7.0.*, although the line numbers will be different.

			regards, tom lane


*** src/backend/utils/adt/numeric.c.orig	Wed Dec  6 21:47:35 2000
--- src/backend/utils/adt/numeric.c	Wed Mar 14 11:50:37 2001
***************
*** 3355,3370 ****
  	init_var(&tmp);
  
  	/* ----------
! 	 * We do it by fiddling around with global_rscale and truncating
! 	 * the result of the division.
  	 * ----------
  	 */
  	save_global_rscale = global_rscale;
  	global_rscale = var2->rscale + 2;
  
  	div_var(var1, var2, &tmp);
  	tmp.rscale = var2->rscale;
- 	tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));
  
  	global_rscale = var2->rscale;
  	mul_var(var2, &tmp, &tmp);
--- 3355,3373 ----
  	init_var(&tmp);
  
  	/* ----------
! 	 * We do this using the equation
! 	 *		mod(x,y) = x - trunc(x/y)*y
! 	 * We fiddle a bit with global_rscale to control result precision.
  	 * ----------
  	 */
  	save_global_rscale = global_rscale;
  	global_rscale = var2->rscale + 2;
  
  	div_var(var1, var2, &tmp);
+ 
+ 	/* do trunc() by forgetting digits to the right of the decimal point */
+ 	tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
  	tmp.rscale = var2->rscale;
  
  	global_rscale = var2->rscale;
  	mul_var(var2, &tmp, &tmp);

In response to

pgsql-bugs by date

Next:From: Tom LaneDate: 2001-03-14 18:40:55
Subject: Re: odbc bug?
Previous:From: Tom LaneDate: 2001-03-14 16:32:01
Subject: Re: Various bugs with PG7.1 8th March snapshot on Solaris 8 INTEL

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group