Re: buildfarm windows checks / tap tests on windows

Hi,

On 2021-03-02 07:48:28 -0500, Andrew Dunstan wrote:
> I don't think speed is the issue. I probably disabled misc-tests on
> drongo and bowerbird (my two animals in question) because I got  either
> instability or errors I was unable to diagnose. I'll go back and take
> another look to narrow this down. It's possible to disable individual tests.

Yea, there was one test that hung in 021_row_visibility.pl which was a
weird perl hang - with weird symptoms. But that should be fixed now.

There's another failure in recoverycheck that I at first thought was the
fault of the aio branch. But it also see it on master - but only on one
of the two machines I use to test. Pretty odd.

t/003_recovery_targets.pl ............ 7/9
# Failed test 'multiple conflicting settings'
# at t/003_recovery_targets.pl line 151.

# Failed test 'recovery end before target reached is a fatal error'
# at t/003_recovery_targets.pl line 177.
t/003_recovery_targets.pl ............ 9/9 # Looks like you failed 2 tests of 9.
t/003_recovery_targets.pl ............ Dubious, test returned 2 (wstat 512, 0x200)
Failed 2/9 subtests

I think it's pretty dangerous if we have a substantial number of tests
that aren't run on windows - I think a lot of us just assume that the
BF would catch windows specific problems...

> > On that last point: Running the tap tests on windows appears to be
> > *excruciatingly* slow. How does anybody develop on windows without a
> > mechanism to actually run tests in parallel?
>
>
> I think most people develop elsewhere and then adapt/test on Windows if
> necessary.

Yea, but even that overstretches my patience by a good bit. I can't deal
with a serial check-world on linux either - I think that's the biggest
differentiator. It's a lot less painful to deal with slow-ish tests if
they do all their slowness concurrently. But that's basically impossible
with vcregress.pl, and the bf etc can't easily do it either until we
have a decent way to see the correct logfiles & output for individual
tests...

> > I think it'd be good if vcregress.pl at least respected PROVE_FLAGS from
> > the environment - it can't currently be passed in for several of the
> > vcregress.pl tests, and it does seem to make things to be at least
> > somewhat less painful.
> +1

K, will send a patch for that in a bit.

> > This makes it even clearer to me that we really need a builtin
> > testrunner that runs tests efficiently *and* debuggable on windows.
> >
>
> "show me the code" :-)

The biggest obstacle on that front is perl. I started to write one, but
hit several perl issues within an hour. I think I might write one in
python, that'd be a lot less painful.

One windows build question I have is why the msvc infrastructure doesn't
accept msys perl in places like this:
guid => $^O eq "MSWin32" ? Win32::GuidGen() : 'FAKE',
If I change them to accept msys perl then the build ends up working.

The reason it'd be nice to accept msys perl is that git for windows
bundles that - and that's already installed on most CI projects...

Greetings,

Andres Freund