From: | Heikki Linnakangas <heikki(dot)linnakangas(at)iki(dot)fi> |
---|---|
To: | pgsql-committers(at)postgresql(dot)org |
Subject: | pgsql: Perform conversion from Python unicode to string/bytes object vi |
Date: | 2012-08-06 11:34:09 |
Message-ID: | E1SyLZZ-00019p-0M@gemulon.postgresql.org |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-committers |
Perform conversion from Python unicode to string/bytes object via UTF-8.
We used to convert the unicode object directly to a string in the server
encoding by calling Python's PyUnicode_AsEncodedString function. In other
words, we used Python's routines to do the encoding. However, that has a
few problems. First of all, it required keeping a mapping table of Python
encoding names and PostgreSQL encodings. But the real killer was that Python
doesn't support EUC_TW and MULE_INTERNAL encodings at all.
Instead, convert the Python unicode object to UTF-8, and use PostgreSQL's
encoding conversion functions to convert from UTF-8 to server encoding. We
were already doing the same in the other direction in PLyUnicode_FromString,
so this is more consistent, too.
Note: This makes SQL_ASCII to behave more leniently. We used to map
SQL_ASCII to Python's 'ascii', which on Python means strict 7-bit ASCII
only, so you got an error if the python string contained anything but pure
ASCII. You no longer get an error; you get the UTF-8 representation of the
string instead.
Backpatch to 9.0, where these conversions were introduced.
Jan Urbański
Branch
------
REL9_1_STABLE
Details
-------
http://git.postgresql.org/pg/commitdiff/31595993901484d24c9ba62428c7abec207dd55e
Modified Files
--------------
src/pl/plpython/expected/plpython_unicode_3.out | 54 -------------
src/pl/plpython/plpython.c | 98 ++++++++++-------------
2 files changed, 44 insertions(+), 108 deletions(-)
From | Date | Subject | |
---|---|---|---|
Next Message | Heikki Linnakangas | 2012-08-06 11:59:43 | Re: Re: [COMMITTERS] pgsql: Fix mapping of PostgreSQL encodings to Python encodings. |
Previous Message | Bruce Momjian | 2012-08-05 03:26:51 | Re: pgsql: Replace pgindent shell script with Perl script. Update perltidy |