Re: TAP test breakage on MacOS X

On Mon, Oct 6, 2014 at 8:15 PM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
> On Thu, 2014-10-02 at 21:18 -0400, Robert Haas wrote:
>> > If none of this gets us closer to an answer, I can try to produce a
>> > patch that produces more details for such failures.
>>
>> A test that fails for no reason that can be gleaned from the output is
>> not an improvement over not having a test at all.
>
> I understand that this isn't great, and it's certainly something I'm
> looking into. But it's like pg_regress saying that psql crashed and
> leaving you to find out why. I don't think saying that the entire
> regression test suite is useless because of that is fair. The TAP tests
> are arguably already much easier to debug than pg_regress ever was.

I looked into this a bit more. This fixes it:

diff --git a/src/test/perl/TestLib.pm b/src/test/perl/TestLib.pm
index 545b2f3..2ffb7eb 100644
--- a/src/test/perl/TestLib.pm
+++ b/src/test/perl/TestLib.pm
@@ -205,6 +205,7 @@ sub program_options_handling_ok
'2>', \$stderr;
ok(!$result, "$cmd with invalid option nonzero exit code");
isnt($stderr, '', "$cmd with invalid option prints
error message");
+ run [ '/usr/bin/true' ];
};
}

Not that I'm recommending that fix, of course.

The problem is that running initdb --not-a-valid-option leaves $? set
to 256, as seems entirely unsurprising. After running the anonymous
block passed to it, Test::Builder::subtest calls Test::Build::finalize
which calls Test::Build::_ending, which sets $real_exit_code to $? -
i.e. 256. That function later throws up if $real_exit_code isn't 0,
hence the failure.

Off-hand, I'm not quite sure why this works for anyone.

--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company