Re: patch: Add JSON datatype to PostgreSQL (GSoC, WIP)

On Tue, Oct 19, 2010 at 11:22 AM, Terry Laurenzo <tj(at)laurenzo(dot)org> wrote:
> Agreed.  BSON was born out of implementations that either lacked arbitrary
> precision numbers or had a strong affinity to an int/floating point way of
> thinking about numbers.  I believe that if BSON had an arbitrary precision
> number type, it would be a proper superset of JSON.
>
> As an aside, the max range of an int in BSON 64bits.  Back to my original
> comment that BSON was "grown" instead of designed, it looks like both the
> 32bit and 64bit integers were added late in the game and that the original
> designers perhaps were just going to store all numbers as double.
>
> Perhaps we should enumerate the attributes of what would make a good binary
> encoding?

I think we should take a few steps back and ask why we think that
binary encoding is the way to go. We store XML as text, for example,
and I can't remember any complaints about that on -bugs or
-performance, so why do we think JSON will be different? Binary
encoding is a trade-off. A well-designed binary encoding should make
it quicker to extract a small chunk of a large JSON object and return
it; however, it will also make it slower to return the whole object
(because you're adding serialization overhead). I haven't seen any
analysis of which of those use cases is more important and why.

I am also wondering how this proposed binary encoding scheme will
interact with TOAST. If the datum is compressed on disk, you'll have
to decompress it anyway to do anything with it; at that point, is
there still going to be a noticeable speed-up from using the binary
encoding?

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company