The real reason why TAP testing isn't ready for prime time

Buildfarm member hamster has failed a pretty significant fraction of
its recent runs in the BinInstallCheck step:
http://buildfarm.postgresql.org/cgi-bin/show_history.pl?nm=hamster&br=HEAD

Since other critters aren't equally distressed, it seems likely that
this is just an out-of-disk-space type of problem. But maybe it's
trying to tell us that there's a genuine platform-specific bug there.
In any case, I challenge anyone to figure out what's happening from
the information available from the buildfarm logs.

I don't know whether this is just that the buildfarm script isn't
collecting data that it should be. But my experiences with the
TAP test scripts haven't been very positive. When they fail, it
takes a lot of digging to find out why. Basically, that entire
mechanism sucks as far as debuggability is concerned.

I think there is a good argument for turning this off in the buildfarm
until there is a better way of identifying and solving problems. It is
not helping us that hamster is red half the time for undiscoverable
reasons. That just conditions people to ignore it, and it may well be
masking real problems that the machine could be finding if it weren't
failing at this step.

regards, tom lane