Re: Inadequate traces in TAP tests

Stephen Frost <sfrost(at)snowman(dot)net> writes:
> * Andrew Dunstan (andrew(dot)dunstan(at)2ndquadrant(dot)com) wrote:
>> ISTM that the test setup and breakdown code, both in individual tests
>> and in PostgresNode.pm should be liberally sprinkled with diag() calls
>> to make it easier to narrow down errors..

> While I'm generally in favor of adding diag() info into the testing for
> when things go wrong, what I don't want to do is increase the amount of
> output that these tests produce without good cause. I really wish there
> was a "quiet" mode for the TAP tests which didn't report anything when
> things are 'ok'.

FWIW, the problem I've got with the TAP tests is that when one fails
in the buildfarm, you've got to dig through megabytes of all-alike-looking
output just to try to determine which one failed; and once you do, you
still know nothing because the script output never really says why it
thinks there was a problem. If you're lucky, you can identify the
postmaster log file(s) corresponding to the failed test script, and then
you can try to guess from the log entries what went wrong.

I am *absolutely* not in favor of adding anything to the scripts' routine
output, because it will just make this problem worse by bloating the
buildfarm logs even more. What I'd like to see is for the scripts to
always report something along the lines of "this is what I got, this is
what I expected to get" --- but only when there is a failure. The less
output there is from a successful test, the better, IMO.

regards, tom lane