Re: Portability issues in TAP tests

On Mon, Jul 21, 2014 at 10:06 AM, Christoph Berg <cb(at)df7cb(dot)de> wrote:
> Re: Noah Misch 2014-07-18 <20140718052625(dot)GA2231360(at)tornado(dot)leadboat(dot)com>
>> Installing a new version of one Perl module is well within the capabilities of
>> buildfarm owners. Saving them the trouble, which in turn means more of them
>> actually activating the TAP tests, might justify the loss. I'd be sad to see
>> the "subtest" use go away, but I lean toward thinking it's for the best.
>>
>> From a technical standpoint, it would be nicest to bundle copies of Test::More
>> and IPC::Run for the test suites to use.
>
> Please not. If no OS packages are available, there's still "cpan IPC::Run"
> to get it installed to $HOME automatically.
>
>> A minor point to add to your list:
>>
>> 5. The TAP suites don't work when $PWD contains spaces.
>
> Here's yet another thing that I think is pretty major:
>
> 6. The tests fail if your $LANG isn't en_something:
>
> ok 1 - vacuumdb -z postgres exit code 0
> not ok 2 - vacuumdb -z: SQL found in server log
>
> # Failed test 'vacuumdb -z: SQL found in server log'
> # at /home/cbe/projects/postgresql/postgresql/9.4/build/../src/test/perl/TestLib.pm line 221.
> # 'LOG: Anweisung: VACUUM (ANALYZE);
> # '
> # doesn't match '(?^:statement: VACUUM \(ANALYZE\);)'
> # Looks like you failed 1 test of 2.
>
> ... repeated a gazillion times. pg_regress unsets LANG and LC_*, the
> perl tests should do also.

While we're complaining...

The tests weren't running for me at all on MacOS X, because I was
missing some prerequisite. So I installed it, and now they promptly
fail:

ok 2 - initdb --version
1..2
ok 1 - initdb with invalid option nonzero exit code
ok 2 - initdb with invalid option prints error message
# Looks like your test exited with 256 just after 2.
not ok 3 - initdb options handling

Unlike the regular regression tests, these tests don't seem to provide
any guidance on where to see the difference between the expected and
actual output, or how to get more details, so that's all I know.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company