control character check in JSON type seems broken

From: Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>
To: pgsql-bugs(at)postgresql(dot)org
Subject: control character check in JSON type seems broken
Date: 2012-06-04 10:41:47
Message-ID: CAEZqfEdbRO8MhH83Fma9uSQhCfWmHpgcbm9Lwucq2HSwCHhbdQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs

Hi,

A Japanese user found strange behavior in JSON type, so I'd like to
share the issue here. He simply tested casting a string literal to json
type, and got an unexpected error when he used a Japanese word as name
and/or value of JSON object.

In the example below, "キー" is a Japanese word which means "key", and
its first letter has byte sequence "0xe3 0x82 0xad" in UTF-8.

postgres=# select '{"キー":"value"}'::json;
ERROR: invalid input syntax for type json
LINE 1: select '{"キー":"value"}'::json;
^
DETAIL: line 1: Character " ・ must be escaped.

With some debugging, I found that the problem is in json_lex_string().

json_lex_string() misjudges that the token "キー" contains naked
(not-escaped) control character, because its first byte is 0xe3 and it's
-29 for signed char interpreting, and it's less than 32. We need to
cast to unsigned char (or use unsigned variable) here.

In addition, error message above seems corrupted in my environment.
Here we check not-escaped control character, so printing it with %c
formatting might break log files. How about using decimal or hex dump
in such cases?

Please see attached patch which contains workaround for this issue.

Regards,
--
Shigeru HANADA

Attachment Content-Type Size
fix_json_check.patch text/plain 1.4 KB

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message pgagarinov 2012-06-04 12:57:42 Calling xlst_process with certain arguments causes server crash
Previous Message Tom Lane 2012-06-03 15:54:29 Re: BUG #6672: Memory leaks in dumputils.c