Re: BUG #15548: Unaccent does not remove combining diacritical characters

From: Hugh Ranalli <hugh(at)whtc(dot)ca>
To: Michael Paquier <michael(at)paquier(dot)xyz>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>, thomas(dot)munro(at)enterprisedb(dot)com, Daniel Verite <daniel(at)manitou-mail(dot)org>, pgsql-bugs(at)lists(dot)postgresql(dot)org
Subject: Re: BUG #15548: Unaccent does not remove combining diacritical characters
Date: 2019-01-10 14:10:43
Message-ID: CAAhbUMMsJ-p4QNk_LEOG3L61Z92o=McXXfy8qA3b7=xXGKuvtw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

On Thu, 10 Jan 2019 at 01:09, Michael Paquier <michael(at)paquier(dot)xyz> wrote:

> OK, committed then. I have yet to study yet the other part of the
> proposal regarding diatritical characters. Patch 3 has a conflict for
> the regression tests, so a rebase would be needed. That's not a big
> deal though to resolve the conflict. I am also a bit confused by the
> newly-generated unaccent.rules. Why nothing shows up for the second
> column (around line 414 for example)? Shouldn't we have mapping
> characters?
>

That concerned me, as well. I have confirmed the lines are not empty. If
you open the file in a text editor (I'm using KDE's Kate), and insert a
standard character at the beginning of one of those lines, the diacritic
then appears, combined with the character you just entered. The only
program I've found that wants to display them on their own is vi (and I
only just thought of trying that).

From what I can tell, this is likely a font issue:

- http://unicode.org/faq/char_combmark.html#12b
-
https://superuser.com/questions/852901/why-are-some-combining-diacritics-shifted-to-the-right-in-some-programs

Hugh

In response to

Browse pgsql-bugs by date

  From Date Subject
Next Message PG Bug reporting form 2019-01-10 15:02:04 BUG #15588: Garbled code when log in
Previous Message PG Bug reporting form 2019-01-10 13:55:24 BUG #15587: Partitions with ALTER TABLE ADD CONSTRAINT

Browse pgsql-hackers by date

  From Date Subject
Next Message Laurenz Albe 2019-01-10 14:56:41 Re: Remove Deprecated Exclusive Backup Mode
Previous Message Arthur Zakirov 2019-01-10 13:00:18 Re: PostgreSQL vs SQL/XML Standards