Re: reducing NUMERIC size for 9.1

On Wed, Aug 4, 2010 at 11:05 AM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> On Fri, Jul 30, 2010 at 9:55 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>> The smallest value for precision which requires 2 numeric_digits is
>>> always 2; and the required number of numeric_digits increases by 1
>>> each time the number of base-10 digits increases by DEC_DIGITS.
>
>> And here is a patch implementing that.
>
> Looks good to me.
>
> One thought --- to make this look more like the typmod-whacking in
> the numeric() function, perhaps the first line of numeric_maximum_size
> ought to be
>
>        if (typemod <= (int32) (VARHDRSZ))
>                return -1;
>
> I think the author of numeric() was concerned that VARHDRSZ might be
> unsigned, in which case the cast-less comparison would do the Wrong
> Thing.  Reference to c.h shows that it's signed, so no bug, but having
> the cast in the comparison seems like good conservative programming.
>
> Or, if you prefer, you can take out the cast in numeric().  I merely
> opine that the two bits of code should match.

*scratches head*

One of those tests uses < and the other uses <=

Which one is wrong?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise Postgres Company