Skip site navigation (1) Skip section navigation (2)

Re: [HACKERS] UNICODE/UTF-8 on win32

From: "John Hansen" <john(at)geeknet(dot)com(dot)au>
To: "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>
Cc: "Tatsuo Ishii" <t-ishii(at)sra(dot)co(dot)jp>, <mha(at)sollentuna(dot)net>,<tgl(at)sss(dot)pgh(dot)pa(dot)us>, <pgsql-hackers-win32(at)postgresql(dot)org>,<pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] UNICODE/UTF-8 on win32
Date: 2005-04-24 14:09:08
Message-ID: 5066E5A966339E42AA04BA10BA706AE56297@rodrick.geeknet.com.au (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-hackers-win32
Ehmm,... No the upper/lower replacements I sent to -hackers

ICU was not me.... Tho for win32 you're better off writing wrapper
classes for the win32 native functions. 

> -----Original Message-----
> From: Bruce Momjian [mailto:pgman(at)candle(dot)pha(dot)pa(dot)us] 
> Sent: Sunday, April 24, 2005 10:50 PM
> To: John Hansen
> Cc: Tatsuo Ishii; mha(at)sollentuna(dot)net; tgl(at)sss(dot)pgh(dot)pa(dot)us; 
> pgsql-hackers-win32(at)postgresql(dot)org; pgsql-hackers(at)postgresql(dot)org
> Subject: Re: [HACKERS] [pgsql-hackers-win32] UNICODE/UTF-8 on win32
> 
> John Hansen wrote:
> > Look at the upper/lower I sent to the list, they should be able to 
> > replace upper/lower for the utf8 encoding.... (and works 
> independent 
> > of locale)..
> 
> You mean ICU?  Yes, it seems like a good approach for 8.1.
> 
> --------------------------------------------------------------
> -------------
> 
> 
> > 
> > ... John
> > 
> > > -----Original Message-----
> > > From: pgsql-hackers-owner(at)postgresql(dot)org
> > > [mailto:pgsql-hackers-owner(at)postgresql(dot)org] On Behalf Of Bruce 
> > > Momjian
> > > Sent: Sunday, April 24, 2005 10:35 PM
> > > To: Tatsuo Ishii
> > > Cc: mha(at)sollentuna(dot)net; tgl(at)sss(dot)pgh(dot)pa(dot)us; 
> > > pgsql-hackers-win32(at)postgresql(dot)org; pgsql-hackers(at)postgresql(dot)org
> > > Subject: Re: [HACKERS] [pgsql-hackers-win32] 
> UNICODE/UTF-8 on win32
> > > 
> > > 
> > > Where are we on this?  As far as I can tell, we never 
> disabled UTF8 
> > > on
> > > Win32 in our code.  The only thing we did do was to disable
> > > UTF8 in pginstaller.  See this FAQ item:
> > > 
> > > 	
> > > 
> http://pginstaller.projects.postgresql.org/faq/FAQ_windows.html#2.6
> > > 
> > > Is the current setup OK?  Should we allow UTF8 on Win32 for 
> > > languages that can use C locale, like Asian languages?
> > > 
> > > --------------------------------------------------------------
> > > -------------
> > > 
> > > Tatsuo Ishii wrote:
> > > > I do understand the problem, but don't undertstand the decision 
> > > > you guys made. The fact that UPPER/LOWER and some other
> > > functions does not
> > > > work in win32 is surely a problem for some languages, but not a 
> > > > problem for otheres. For example, Japanese (and 
> probably Chinese 
> > > > and
> > > > Korean) does not have a concept upper/lower. So the fact
> > > UPPER/LOWER
> > > > does not work with UTF-8/win32 is not problem for Japanese (and 
> > > > for some other languages). Just using C locale with UTF-8 is 
> > > > enough in this case.
> > > > 
> > > > In summary, I think you guys are going to overkill the 
> multibyte 
> > > > support functionality on UTF-8/win32 because of the 
> fact that some 
> > > > langauges do not work.
> > > > 
> > > > Same thing can be said to EUC-JP, EUC-CN and EUC-KR and so
> > > on as well.
> > > > 
> > > > I strongly object the policy to try to unconditionaly disable 
> > > > UTF-8 support on win32.
> > > > --
> > > > Tatsuo Ishii
> > > > 
> > > > From: "Magnus Hagander" <mha(at)sollentuna(dot)net>
> > > > Subject: RE: [pgsql-hackers-win32] UNICODE/UTF-8 on win32
> > > > Date: Sat, 1 Jan 2005 14:48:04 +0100
> > > > Message-ID: 
> > > > <6BCB9D8A16AC4241919521715F4D8BCE4764A4(at)algol(dot)sollentuna(dot)se>
> > > > 
> > > > > UNICODE/UTF-8 does not work on the win32 server. The
> > > reason is that
> > > > > strcoll() and friends don't work with it. To support it
> > > on win32, it
> > > > > needs to be converted to UTF16 and use the wide-character
> > > versions
> > > > > of the fucntion. Which we do not do.
> > > > > (See
> > > > > 
> > > 
> http://archives.postgresql.org/pgsql-hackers-win32/2004-11/msg00036.
> > > > > php
> > > > > and
> > > > > 
> > > 
> http://archives.postgresql.org/pgsql-hackers-win32/2004-12/msg00106.
> > > > > php)
> > > > > 
> > > > > 
> > > > > I don't *think* we need to disable ito n the client. 
> AFAIK, the 
> > > > > client interfaces don't use any of these functions, and I've 
> > > > > seen reports of people using that long before we had a native
> > > win32 server.
> > > > > 
> > > > > 
> > > > > //Magnus
> > > > > 
> > > > > 
> > > > > >-----Original Message-----
> > > > > >From: Tatsuo Ishii [mailto:t-ishii(at)sra(dot)co(dot)jp]
> > > > > >Sent: den 1 januari 2005 01:10
> > > > > >To: tgl(at)sss(dot)pgh(dot)pa(dot)us
> > > > > >Cc: Magnus Hagander; pgsql-hackers-win32(at)postgresql(dot)org
> > > > > >Subject: Re: [pgsql-hackers-win32] UNICODE/UTF-8 on win32
> > > > > >
> > > > > >
> > > > > >Sorry, but I don't subscribe to pgsql-hackers-win32 list. 
> > > > > >What's the problem here?
> > > > > >--
> > > > > >Tatsuo Ishii
> > > > > >
> > > > > >> "Magnus Hagander" <mha(at)sollentuna(dot)net> writes:
> > > > > >> > We know it's broken and won't be fixed for 8.0.
> > > > > >> 
> > > > > >> > If we just #ifndef WIN32 the definitions in
> > > > > >utils/mb/encnames.c it won't
> > > > > >> > be possible to select that encoding, right? Will 
> that have
> > > > > >any other
> > > > > >> > unwanted effects (such as breaking client encodings)? If
> > > > > >not, I suggest
> > > > > >> > this is done.
> > > > > >> 
> > > > > >> I believe the subscripts in those arrays have to match the 
> > > > > >> encoding enum type, so you can't just ifdef out
> > > individual entries.
> > > > > >> 
> > > > > >> > (Or perhaps something can be done in
> > > pg_valid_server_encoding?)
> > > > > >> 
> > > > > >> Making the valid_server_encoding function reject 
> it might work.
> > > > > >> Tatsuo-san would know for sure.
> > > > > >> 
> > > > > >> Should we also reject it as a client encoding, or does
> > > that work OK?
> > > > > >> 
> > > > > >> 			regards, tom lane
> > > > > >> 
> > > > > >
> > > > > 
> > > > 
> > > > ---------------------------(end of
> > > > broadcast)---------------------------
> > > > TIP 1: subscribe and unsubscribe commands go to 
> > > > majordomo(at)postgresql(dot)org
> > > > 
> > > 
> > > -- 
> > >   Bruce Momjian                        |  http://candle.pha.pa.us
> > >   pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
> > >   +  If your life is a hard drive,     |  13 Roberts Road
> > >   +  Christ can be your backup.        |  Newtown Square, 
> > > Pennsylvania 19073
> > > 
> > > ---------------------------(end of
> > > broadcast)---------------------------
> > > TIP 9: the planner will ignore your desire to choose an 
> index scan 
> > > if your
> > >       joining column's datatypes do not match
> > > 
> > > 
> > 
> 
> -- 
>   Bruce Momjian                        |  http://candle.pha.pa.us
>   pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 359-1001
>   +  If your life is a hard drive,     |  13 Roberts Road
>   +  Christ can be your backup.        |  Newtown Square, 
> Pennsylvania 19073
> 
> 

pgsql-hackers by date

Next:From: Tom LaneDate: 2005-04-24 15:41:17
Subject: Re: Constant WAL replay
Previous:From: Neil ConwayDate: 2005-04-24 13:36:51
Subject: Re: Woo hoo ... a whole new set of compiler headaches!!

pgsql-hackers-win32 by date

Next:From: Daniel BergerDate: 2005-04-24 15:46:21
Subject: libpq-fs.h missing from windows installer
Previous:From: Bruce MomjianDate: 2005-04-24 12:50:28
Subject: Re: [HACKERS] UNICODE/UTF-8 on win32

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group