Re: buildfarm / handling (undefined) locales

On 13.5.2014 20:58, Tom Lane wrote:
> Tomas Vondra <tv(at)fuzzy(dot)cz> writes:
>> a few days ago I switched magpie into an LXC container, and while
>> fixinig unrelated issue there, I noticed that although the tests seemed
>> to run, some of the results are actually rubbish because of missing locales.
>
>> So for example when the system misses cs_CZ.WIN-1250 (i.e. czech locale
>> with windows codepage 1250), initdb actually did this
>
>> The files belonging to this database system will be owned by user
>> "pgbuild".
>> This user must also own the server process.
>
>> initdb: invalid locale name ""
>> initdb: invalid locale name ""
>> initdb: invalid locale name ""
>> initdb: invalid locale name ""
>> initdb: invalid locale name ""
>> initdb: invalid locale name ""
>> The database cluster will be initialized with locale "C".
>> The default database encoding has accordingly been set to
>> "SQL_ASCII".
>> The default text search configuration will be set to "english".
>
> Hm, I'm a bit confused as to what you actually did here. The "invalid
> locale name" bleats seem to indicate that no --locale or --lc_xxx options
> were given on the command line; correct? If so the issue is presumably
> that the environment variable(s) were set to incorrect values. While
> we *could* abort in that situation, I've never heard of any program
> that did; the normal response is to silently ignore the environment
> variables and use C locale. We're not being exactly silent about it
> but I think the outcome is the expected one.
>
> There is a comment in the code about this:
>
> /* should we exit here? */
> if (res == NULL)
> fprintf(stderr, _("%s: invalid locale name \"%s\"\n"),
> progname, locale);
>
> but I think what's being questioned is whether an incorrect locale
> name *given as a command line argument* should result in an abort.
> That might be a good idea, but it looks like it'd require some
> restructuring of the code to make it possible to distinguish the
> case from bad-environment.

Hmmmm. Actually the logs above were generated "manually" by setting LANG
to an incorrect value on my desktop machine, i.e. something like this:

export LANG="cs_CZ.WIN-1250"
pg_ctl -D /tmp/test init

I did this because the logs from the buildfarm animal contain the
messages in czech, which makes them rather unsuitable for english speakers:

http://www.pgbuildfarm.org/cgi-bin/show_stage_log.pl?nm=magpie&dt=2014-04-17%2000%3A07%3A36&stg=initdb-cs_CZ.WIN-1250

I suppose that's why the locale name is empty (it simply gets empty
string from the environment), while the buildfarm client uses a
command-line option to pass the name.

Also, I now noticed that while the example I posted has to C.SQL_ASCII,
the buildfarm then uses cs_CZ.UTF-8. I guess this possible thanks to the
'--locale' option.

If it was up to me, I'd vote to fail in such cases. I find it confusing
(and a bit 'automagic') to receive invalid locale but decide 'the user
surely meant C with SQL_ASCII'. I've actually had to deal with a few
production installations that were installed like this, and it was PITA
to fix.

Anyway, that's not what I was proposing here. I merely think we should
do a simple 'did we actually get the right locale' check somewhere in
the buildfarm client.

>> Yeah, not really what we were shooting for. I've fixed this by defining
>> the missing locales, and indeed - magpie now fails in plpython tests.
>
> I saw that earlier today (tho right now the buildfarm server seems to
> not be responding :-(). Probably we should use some more-widely-used
> character code in that specific test?

What do you mean by 'not responding'? The tests were not running for ~2
days because of LXC/SELinux issue, causing cron failures.

Anyway, using a more-widely used character is probably the best fix
possible here.

regards
Tomas