Skip site navigation (1) Skip section navigation (2)

Re: pl/perl and utf-8 in sql_ascii databases

From: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>
To: tgl(at)sss(dot)pgh(dot)pa(dot)us
Cc: badalex(at)gmail(dot)com, alvherre(at)commandprompt(dot)com, cb(at)df7cb(dot)de, pgsql-hackers(at)postgresql(dot)org
Subject: Re: pl/perl and utf-8 in sql_ascii databases
Date: 2012-07-03 08:59:38
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-hackers
Hello, Here is regression test runs on pg's also built with
cygwin-gcc and VC++.

The patches attached following,

- plperl_sql_ascii-4.patch         : fix for pl/perl utf8 vs sql_ascii
- plperl_sql_ascii_regress-1.patch : regression test for this patch.
                                     I added some tests on encoding to this.

I will mark this patch as 'ready for committer' after this.

For the continuity of the behavior for sql_ascii and the chars
like \x80, It might be better if the main patch is back ported
into 9.1 and 9.2. New regression tests seems to have less
necessity to do since it has not been there from the first..

This regression test runs for all of
  Build with gcc3 / Linux(CentOS6.2-64)
  Built with Cygwin gcc3 / Windows7-64
  Built with VC++2008 / ActivePerl5.12 / Windows7-64


I've been stuck in mud trying to plperl work on windows
environment. I saw many messages complaining that plperl wouldn't
be built to work. For the convenience of those and myself, I
describe the process of building postgresql with plperl on
Windows with cygwin and VC++ I've done below.

> Ok. Since there found to be only two patterns in the regression
> test. The fancy thing is no more needed. I will unfold them and
> make sure to work on mingw build environment.
> And for one more environment, on the one with VC++.. I'll need a
> bit longer time to make out what does.

I could understand what you meant after I managed to build plperl
to run properly. reads $REGRESS in GNUmakefile so
variable substitution of make does not work on Windows'
regression. I resolved this problem by copying plperl_lc_*.out
files into plperl_lc.out before it runs pg_regress.

> - The main patch fixes the sql-ascii handling itself shoud ported
>   into 9.2 and 9.1. Someone shoud work for this. (me?)


> - The remainder of the patch whic fixes the easy fixable leakes
>   of palloc'ed memory won't be ported into 9.1. This is only for
>   9.3dev.
> - The patch for 9.3dev will be provided with the new regression
>   test. It will be easily ported into 9.1 and 9.2 and there seems
>   to be no problem technically, but a bit unsure from the other
>   points of view...

What should I do for this?


Addition - Building Windows binary for plperl

NOTE: This is ONE example I tried and turned out a success.

A. Cygwin

 Versions: Windows 7 64bit
           Cygwin 1.7.15
           gcc 3.4.4 (cygwin-server is running)

 1. Build perl aside system-installed one
    perl-5.16.0$ export PATH=/usr/local/bin:/bin:/usr/bin:/usr/sbin
    perl-5.16.0$ ./Configure --
    perl-5.16.0$ ./Configure -d
    perl-5.16.0$ make
    perl-5.16.0$ make install

  - The first line needed to avoid the Makefile of perl stops by
    parens in search path.

 2. Build postgresql with --with-perl
    pg93dev$ ./configure --with-perl
    pg93dev$ make all

 3. Run the regression test for plperl
    pg93dev/src/pl/plperl$ make check
    pg93dev/src/pl/plperl$ make check ENCODING=sql-ascii

B. VC++   

 Versions: Windows 7 64bit
           Microsoft Visual C++ 2008 Express Edition
           Active Perl v6.12.4 x86

 1. Install Active Perl normally.  Assuming the install location
    is "c:\Perl" and I did all operation on cmd.exe after this.

 2. Create in src/tools/msvc
    pg> cd src\tools\msvc
    msvc> copy
    .... Edit as follows

    -   perl=>undef, 			# --with-perl
    +   perl=>'c:\Perl', 			# --with-perl

 3. Build it

    msvc> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat
    msvc> build
    msvc> install c:\pgsql

 4. Run the regression tests

    4.1 Create test database and run postgres for UTF8 test

    A> set PATH=c:\pgsql\bin;c:\pgsql\lib;%PATH%
    A> initdb -D <pgdata dir> --no-locale --encoding=utf8
    A> postgres -D <pgdata dir>

    ... on another cmd.exe

    B> set PATH=c:\pgsql\bin;c:\pgsql\lib;%PATH%
    pg> cd src\tools\msvc
    msvc> vcregress plcheck

    4.2 Run regression test for SQL-ASCII.
    A> (delete <pgdata dir> and its contents.)
    A> initdb -D <pgdata dir> --no-locale --encoding=sql_ascii
    A> postgres -D <pgdata dir>
    ... same as 4.1 here after...

Kyotaro Horiguchi
NTT Open Source Software Center

== My e-mail address has been changed since Apr. 1, 2012.

Attachment: plperl_sql_ascii-4.patch
Description: text/x-patch (5.7 KB)
Attachment: plperl_sql_ascii_regress-1.patch
Description: text/x-patch (8.4 KB)

In response to


pgsql-hackers by date

Next:From: Dimitri FontaineDate: 2012-07-03 09:13:11
Subject: Re: Event Triggers reduced, v1
Previous:From: Mark KirkwoodDate: 2012-07-03 08:34:09
Subject: Re: xlog filename formatting functions in recovery

Privacy Policy | About PostgreSQL
Copyright © 1996-2018 The PostgreSQL Global Development Group