Re: Reducing the overhead of NUMERIC data

On Thu, 2005-11-03 at 10:32 -0500, Tom Lane wrote:
> I'd feel a lot happier about this if we could keep the dynamic range
> up to, say, 10^512 so that it's still true that NUMERIC can be a
> universal parse-time representation. That would also make it even
> more unlikely that anyone would complain about loss of functionality.
>
> To do that we'd need 8 bits for weight (-128..127 for a base-10K
> exponent is enough) but we need 9 bits for dscale which does not
> quite fit. I think we could make it go by cramming the sign and
> the high-order dscale bit into the first NumericDigit --- the
> digit itself can only be 0..9999 so there are a couple of bits
> to spare. This probably *would* slow down packing and unpacking of
> numerics, but just by a couple lines of C. Arguably the net reduction
> in I/O costs would justify that.

I've got a working version of the code using the above scheme, with
these additional wrinkles:

NaN is indicated by weight=-128, giving a dynamic range of 10^508.

Zeroes are fully compressed, except when the Scale > 255. In that case,
the first digit is present to signify the presence of the high order
Scale bit.

Comments?

Once 8.1 is released, I'll go back and see if I can improve the coding
in a few days with fresh eyes, then submit a patch.

Best Regards, Simon Riggs