Re: jsonb, unicode escapes and escaped backslashes

On Tue, Jan 27, 2015 at 03:56:22PM -0500, Tom Lane wrote:
> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > On 01/27/2015 02:28 PM, Tom Lane wrote:
> >> Well, we can either fix it now or suffer with a broken representation
> >> forever. I'm not wedded to the exact solution I described, but I think
> >> we'll regret it if we don't change the representation.

> So at this point I propose that we reject \u0000 when de-escaping JSON.

I would have agreed on 2014-12-09, and this release is the last chance to make
such a change. It is a bold wager that could pay off, but -1 from me anyway.
I can already envision the blog post from the DBA staying on 9.4.0 because
9.4.1 pulled his ability to store U+0000 in jsonb. jsonb was *the* top-billed
9.4 feature, and this thread started with Andrew conveying a field report of a
scenario more obscure than storing U+0000. Therefore, we have to assume many
users will notice the change. This move would also add to the growing
evidence that our .0 releases are really beta(N+1) releases in disguise.

> Anybody who's seriously unhappy with that can propose a patch to fix it
> properly in 9.5 or later.

Someone can still do that by introducing a V2 of the jsonb binary format and
preserving the ability to read both formats. (Too bad Andres's proposal to
include a format version didn't inform the final format, but we can wing it.)
I agree that storing U+0000 as 0x00 is the best end state.

> We probably need to rethink the re-escaping behavior as well; I'm not
> sure if your latest patch is the right answer for that.

Yes, we do. No change to the representation of U+0000 is going to fix the
following bug, but that patch does fix it:

[local] test=# select
test-# $$"\\u05e2"$$::jsonb = $$"\\u05e2"$$::jsonb,
test-# $$"\\u05e2"$$::jsonb = $$"\\u05e2"$$::jsonb::text::jsonb;
?column? | ?column?
----------+----------
t | f
(1 row)