Re: tests and meson - test names and file locations

On 12.08.22 18:29, Andres Freund wrote:
>> I don't really understand which problem this solves and how. Sure, the test
>> output is somewhat complex, but I know where it is and I've never found
>> myself wishing it to be somewhere else.
> I'd like the buildfarm and CI a) use parallelism to run tests (that's why the
> BF is slow) b) show the logfiles for exactly the failed test ([1]). We can of
> course iterate through the whole directory tree, somehow identify which log
> files are for which test, and then select the log files for the failed
> tests. But that's much easier to do then when you have a uniform directory
> hierarchy, where you can test which tests have failed based on the filesystem
> alone.

My initial experiences with testing under meson is that it's quite
fragile and confusing (unlike the building, which is quite robust and
understandable). Some of that is the fault of meson, some of that is
our implementation. Surely this can be improved over time, but my
experience has been that it's not there yet.

The idea that we are going to move all the test output files somewhere
else at the same time is not appealing to me. The combination of
fragile plus can't find the diagnostics is not a good one.

Now, this is my experience; others might have different ones.

Also, is there anything in these proposed changes that couldn't also be
applied to the old build system? We are going to be running them in
parallel for some time. It would be good if one doesn't have to learn
two entirely different sets of testing interfaces.