Re: [PATCH] hstore: Fix parsing on Mac OS X: isspace() is locale specific

From: Michael Paquier <michael(at)paquier(dot)xyz>
To: Evan Jones <evan(dot)jones(at)datadoghq(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCH] hstore: Fix parsing on Mac OS X: isspace() is locale specific
Date: 2023-06-12 00:17:53
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Tue, Jun 06, 2023 at 10:16:09AM -0400, Evan Jones wrote:
> I did a quick look at the places found with "git grep isspace" yesterday. I
> agree with the comment from commit 9ae2661: "I've left alone isspace()
> calls in places that aren't really expecting any non-ASCII input
> characters, such as float8in()." There are a number of other calls where I
> think it would likely be safe, and possibly even a good idea, to replace
> isspace() with scanner_isspace(). However, I couldn't find any where I
> could cause a bug like the one I hit in hstore parsing.

Yes, I agree with this feeling. Like 9ae2661, I can't get really
excited about plastering more of that, especially if it were for
timezone value input or dictionary options. One area with a new
isspace() since 2017 is multirangetypes.c, but it is just a copy of

> Original mailing list post for commit 9ae2661 in case it is helpful for
> others:

I have reproduced the original problem reported on macOS 13.4, which
is close to the top of what's available.

Passing to pg_regress some options to use something else than UTF-8
leads to a failure in the tests, so we need a split like
fussyztrmatch to test that:
REGRESS_OPTS='--encoding=SQL_ASCII --no-locale' make check

An other error pattern without a split could be found on Windows, as
select E'key\u0105=>value'::hstore;
- hstore
- "keyÄ…"=>"value"
-(1 row)
+ERROR: character with byte sequence 0xc4 0x85 in encoding "UTF8" has
no equivalent in encoding "WIN1252"
+LINE 1: select E'key\u0105=>value'::hstore;

We don't do that for unaccent, actually, leading to similar failures..
I'll launch a separate thread about that shortly.

With that fixed, the fix has been applied and backpatched. Thanks for
the report, Evan!

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2023-06-12 00:40:52 Re: v16 fails to build w/ Visual Studio 2015
Previous Message Michael Paquier 2023-06-12 00:04:01 Re: [PATCH] pg_regress.c: Fix "make check" on Mac OS X: Pass DYLD_LIBRARY_PATH