Re: buildfarm + meson

On 2023-02-23 06:27:23 -0500, Andrew Dunstan wrote:
>
> On 2023-02-22 We 20:20, Andres Freund wrote:
> >
> > > There is work to do to make sure we pick up the right log files, and maybe
> > > adjust a module or two. I have adopted a design where instead of trying to
> > > know a lot about the testing regime the client needs to know a lot less.
> > > Instead, it gets meson to tell it the set of tests. I will probably work on
> > > enabling some sort of filter, but I think this makes things more
> > > future-proof. I have stuck with the design of making testing fairly
> > > fine-grained, so each suite runs separately.
> > I don't understand why you'd want to run each suite separately. Serially
> > executing the test takes way longer than doing so in parallel. Why would we
> > want to enforce that?
> >
> > Particularly because with meson the tests log files and the failed tests can
> > directly be correlated? And it should be easy to figure out which log files
> > need to be kept, you can just skip the directories in testrun/ that contain
> > test.success.
> >
>
> We can revisit that later. For now I'm more concerned with getting a working
> setup.

My fear is that this ends up being entrenched in the design and hard to change
later.

> The requirements of the buildfarm are a bit different from those of a
> developer, though. Running things in parallel can make things faster, but
> that can also increase the compute load.

Sure, I'm not advocating to using a [high] concurrency by default.

> Also, running things serially makes it easier to report a failure stage that
> pinpoints the test that encountered an issue.

You're relying on running tests in a specific order. Instead you can also just
run tests in parallel and check test status in order and report the first
failed test in that order.

> But like I say we can come
> back to this.

>
> > > On a Windows instance, fairly similar to what's running drongo, I can get a
> > > successful build with meson+VS2019, but I'm getting an error in the
> > > regression tests, which don't like setting lc_time to 'de_DE'. Not sure
> > > what's going on there.
> > Huh, that's odd.
>
>
> See my reply to Michael for details

I suspect the issue might be related to this:

+ local %ENV = (PATH => $ENV{PATH}, PGUSER => $ENV{PGUSER});
+ @makeout=run_log("meson test --logbase checklog --print-errorlogs --no-rebuild -C $pgsql --suite setup --suite regress");

Greetings,

Andres Freund