Re: Unicode normalization SQL functions

On 2020-03-26 08:25, Peter Eisentraut wrote:
> On 2020-03-24 10:20, Peter Eisentraut wrote:
>> Now I have some concerns about the size of the new table in
>> unicode_normprops_table.h, and the resulting binary size. At the very
>> least, we should probably make that #ifndef FRONTEND or something like
>> that so libpq isn't bloated by it unnecessarily. Perhaps there is a
>> better format for that table? Any ideas?
>
> I have figured this out. New patch is attached.
>
> First, I have added #ifndef FRONTEND, as mentioned above, so libpq isn't
> bloated. Second, I have changed the lookup structure to a bitfield, so
> each entry is only 32 bits instead of 64. Third, I have dropped the
> quickcheck tables for the NFD and NFKD forms. Those are by far the
> biggest tables, and you still get okay performance if you do the
> normalization check the long way, since we don't need the recomposition
> step on those cases, which is by far the slowest part. The main use
> case of all of this, I expect, is to check for NFC normalization, so
> it's okay if the other variants are not optimized to the same extent.

I have committed this version.

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services