Re: buildfarm windows checks / tap tests on windows

On 3/1/21 3:07 PM, Andres Freund wrote:
> Hi,
>
> As part of trying to make the aio branch tests on cirrus CI pass with
> some tap tests I noticed that "src/tools/msvc/vcregress.pl recoverycheck"
> hangs. A long phase of remote debugging later I figured out that that's
> not a fault of the aio branch - it also doesn't pass on master (fixed in
> [1]).
>
> Which confused me - shouldn't the buildfarm have noticed? But it seems
> that all msvc animals either don't have tap tests enabled or they
> disable 'misc-checks' which in turn is what the buildfarm client uses to
> trigger all the 'recovery' checks.
>
> It seems we're not just skipping recovery, but also e.g. tap tests in
> contrib, all the tests in src/test/modules/...
>
> Andrew, what's the reason for that? Is it just that they hung at some
> point? Were too slow?

I don't think speed is the issue. I probably disabled misc-tests on
drongo and bowerbird (my two animals in question) because I got  either
instability or errors I was unable to diagnose. I'll go back and take
another look to narrow this down. It's possible to disable individual tests.

>
>
> On that last point: Running the tap tests on windows appears to be
> *excruciatingly* slow. How does anybody develop on windows without a
> mechanism to actually run tests in parallel?

I think most people develop elsewhere and then adapt/test on Windows if
necessary.

>
> I think it'd be good if vcregress.pl at least respected PROVE_FLAGS from
> the environment - it can't currently be passed in for several of the
> vcregress.pl tests, and it does seem to make things to be at least
> somewhat less painful.

+1

>
>
> This makes it even clearer to me that we really need a builtin
> testrunner that runs tests efficiently *and* debuggable on windows.
>

"show me the code" :-)

cheers

andrew

--
Andrew Dunstan
EDB: https://www.enterprisedb.com