Re: jsonb format is pessimal for toast compression

Stephen Frost <sfrost(at)snowman(dot)net> writes:
> I agree that we need to avoid changing jsonb's on-disk representation.

... post-release, I assume you mean.

> Have I missed where a good suggestion has been made about how to do that
> which preserves the binary-search capabilities and doesn't make the code
> much more difficult?

We don't have one yet, but we've only been thinking about this for a few
hours.

> Trying to move the header to the end just for the
> sake of this doesn't strike me as a good solution as it'll make things
> quite a bit more complicated. Is there a way we could interleave the
> likely-compressible user data in with the header instead?

Yeah, I was wondering about that too, but I don't immediately see how to
do it without some sort of preprocessing step when we read the object
(which'd be morally equivalent to converting a series of lengths into a
pointer array). Binary search isn't going to work if the items it's
searching in aren't all the same size.

Having said that, I am not sure that a preprocessing step is a
deal-breaker. It'd be O(N), but with a pretty darn small constant factor,
and for plausible sizes of objects I think the binary search might still
dominate. Worth investigation perhaps.

regards, tom lane