From: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
---|---|
To: | Kemal Bayram <kemal(at)corenetworks(dot)co(dot)uk> |
Cc: | pgsql-bugs(at)postgresql(dot)org |
Subject: | Re: modulo operator bug? |
Date: | 2001-03-14 16:56:44 |
Message-ID: | 27075.984589004@sss.pgh.pa.us |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-bugs |
> Kemal Bayram <kemal(at)corenetworks(dot)co(dot)uk> writes:
>> from psql:
>> kemal=# select 0.99 % 0.50;
>> ?column?
>> ----------
>> 0.04
>> (1 row)
>> Am I missing something fundemental here or is this indeed a bug?
> Looks like a bug to me too. mod_var shouldn't be using tmp.rscale,
> I think.
OK, fixed for 7.1. If you need a fix in 7.0.*, the attached patch should
apply to 7.0.*, although the line numbers will be different.
regards, tom lane
*** src/backend/utils/adt/numeric.c.orig Wed Dec 6 21:47:35 2000
--- src/backend/utils/adt/numeric.c Wed Mar 14 11:50:37 2001
***************
*** 3355,3370 ****
init_var(&tmp);
/* ----------
! * We do it by fiddling around with global_rscale and truncating
! * the result of the division.
* ----------
*/
save_global_rscale = global_rscale;
global_rscale = var2->rscale + 2;
div_var(var1, var2, &tmp);
tmp.rscale = var2->rscale;
- tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + tmp.rscale + 1));
global_rscale = var2->rscale;
mul_var(var2, &tmp, &tmp);
--- 3355,3373 ----
init_var(&tmp);
/* ----------
! * We do this using the equation
! * mod(x,y) = x - trunc(x/y)*y
! * We fiddle a bit with global_rscale to control result precision.
* ----------
*/
save_global_rscale = global_rscale;
global_rscale = var2->rscale + 2;
div_var(var1, var2, &tmp);
+
+ /* do trunc() by forgetting digits to the right of the decimal point */
+ tmp.ndigits = MAX(0, MIN(tmp.ndigits, tmp.weight + 1));
tmp.rscale = var2->rscale;
global_rscale = var2->rscale;
mul_var(var2, &tmp, &tmp);
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2001-03-14 18:40:55 | Re: odbc bug? |
Previous Message | Tom Lane | 2001-03-14 16:32:01 | Re: Various bugs with PG7.1 8th March snapshot on Solaris 8 INTEL |