Re: Implementing full UTF-8 support (aka supporting 0x00)

From: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
To: Craig Ringer <craig(at)2ndquadrant(dot)com>
Cc: Álvaro Hernández Tortosa <aht(at)8kdata(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Implementing full UTF-8 support (aka supporting 0x00)
Date: 2016-08-03 21:00:02
Message-ID: CAEepm=2_oauRJW=K63cvbk4ye8VK70fp+Ldpcpu+uMHS6Td1CQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Aug 4, 2016 at 5:16 AM, Craig Ringer <craig(at)2ndquadrant(dot)com> wrote:
> On 3 August 2016 at 22:54, Álvaro Hernández Tortosa <aht(at)8kdata(dot)com> wrote:
>> What would it take to support it? Isn't the varlena header propagated
>> everywhere, which could help infer the real length of the string? Any
>> pointers or suggestions would be welcome.
>
>
> One of the bigger pain points is that our interaction with C library
> collation routines for sorting uses NULL-terminated C strings. strcoll,
> strxfrm, etc.

That particular bit of the problem would go away if this ever happened:

https://wiki.postgresql.org/wiki/Todo:ICU

ucoll_strcoll takes explicit lengths (though optionally accepts -1 for
null terminated mode).

http://userguide.icu-project.org/strings#TOC-Using-C-Strings:-NUL-Terminated-vs.-Length-Parameters

--
Thomas Munro
http://www.enterprisedb.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2016-08-03 21:13:08 Re: Parallel tuplesort (for parallel B-Tree index creation)
Previous Message Robert Haas 2016-08-03 20:52:34 pgsql: Prevent "snapshot too old" from trying to return pruned TOAST tu