Re: [HACKERS] Problems on NUMERIC

>
> > First I wonder why the can_coerce... stuff is #if'd out of
> > parse_relation.c?
>
> Oh! That looks like my style of #if FALSE, but I can't recall why it is
> that way. Will look at it. Does it work to just substitute an #if TRUE?
> Perhaps I had it disabled during debugging, but...

Hmmm - elog(ERROR, "Type %s(%d) can be coerced to... looks
like debugging code for me. Maybe you wanted to elog(DEBUG...
?

>
> > How do other databases handle this problem. How is the
> > precision of a numeric result defined?
>
> I've enclosed some snippets from my SQL92 2nd Draft Standard doc. It
> gives you a lot of latitude :)
>

Thanks! That helps alot!

>
> d) The precision and scale of the result of division is
> implementation-defined.

I love those definitions :-)

So I'll make the display scale of a division

min( max(S1, S2), SLIMIT)

and the internal result scale

min( RMINIMUM, max(R1, R2) + 2, RLIMIT)

where S1 and S2 are the display scales of the two operands,
R1 and R2 are the internal present scales and SLIMIT, RLIMIT
is the implementation-defined maximum allowed scale at all
(what about 4000 for SLIMIT ?). The RMINIMUM is 8 to have
anything at least computed internal with 8 digits after the
decimal point (because the defaults for NUMERIC are precision
30 scale 6).

If then the result is assigned to another tuples attribute,
numeric(num,typmod) will be called and do the rounding with
the scale defined in typmod. If numeric_out(num) is called
for it, it will be output rounded to the above display scale.

With 'round(att1, 500) / att2' someone can then get the
result with 500 digits scale.

This way it is flexible enough but not to much wasted
computing is done.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#======================================== jwieck(at)debis(dot)com (Jan Wieck) #