Re: longfin and tamandua aren't too happy but I'm not sure why

Hi,

On 2022-10-01 11:14:20 -0500, Justin Pryzby wrote:
> I just tried this, which works fine at least for v11-v14:
> | git checkout origin/REL_15_STABLE .cirrus.yml src/tools/ci
>
> https://cirrus-ci.com/task/5742859943936000 v15a
> https://cirrus-ci.com/task/6725412431593472 v15b
> https://cirrus-ci.com/task/5105320283340800 v13
> https://cirrus-ci.com/task/4809469463887872 v12
> https://cirrus-ci.com/task/6659971021537280 v11

Cool, thanks for trying that! I wonder if there's any problems on other
branches...

> (I still suggest my patches to run all tests using vcregress. The number of
> people who remember that, for v15, cirrusci runs incomplete tests is probably
> fewer than five.)
> https://www.postgresql.org/message-id/20220623193125.GB22452%40telsasoft.com
> https://www.postgresql.org/message-id/20220828144447.GA21897%40telsasoft.com

Andrew, the defacto maintainer of src/tools/msvc, kind of NACKed those. But
the reasoning might not hold with vcregress being on life support.

OTOH, to me the basic advantage is to have *any* CI coverage. We don't need to
put the bar for the backbranches higher than were we were at ~2 weeks ago.

> If cirrusci were backpatched, it'd be kind of nice to use a ccache key
> that includes the branch name (but maybe the overhead of compilation is
> unimportant compared to the workload induced by cfbot).

Hm. The branch name in general sounds like it might be too broad, particularly
for cfbot. I think we possibly should just put the major version into
.cirrus.yml and use that as the cache key. I think that'd also solve some of
the "diff against what" arguments we've had around your CI improvements.

> A gripe from me: the regression.diffs and other logs from the SQL regression
> tests are in a directory called "main" (same for "isolation"). I imagine I
> won't be the last person to spend minutes looking through the list of test dirs
> for the entry called "regress", conclude that it's inexplicably absent, and
> locate it only after reading src/test/regress/meson.build.

I'd have no problem renaming main/isolation to isolation/isolation and
main/regress to pg_regress/regress or such.

FWIW, if you add --print-errorlogs meson test will show you the output of just
failed tests, which for pg_regress style tests will include the path to
regression.diffs:

...
The differences that caused some tests to fail can be viewed in the
file "/srv/dev/build/m/testrun/cube/regress/regression.diffs". A copy of the test summary that you see
above is saved in the file "/srv/dev/build/m/testrun/cube/regress/regression.out".

It's too bad the default of --print-errorlogs can't be changed.

Unfortunately we don't print something as useful in the case of tap tests. I
wonder if we should do something like

diff --git i/src/test/perl/PostgreSQL/Test/Utils.pm w/src/test/perl/PostgreSQL/Test/Utils.pm
index 99d33451064..acc18ca7c85 100644
--- i/src/test/perl/PostgreSQL/Test/Utils.pm
+++ w/src/test/perl/PostgreSQL/Test/Utils.pm
@@ -239,6 +239,8 @@ END
#
# Preserve temporary directories after (1) and after (2).
$File::Temp::KEEP_ALL = 1 unless $? == 0 && all_tests_passing();
+
+ diag("test logfile: $test_logfile");
}

=pod

Potentially doing so only if $? != 0.

This would make the output for a failing test end like this:
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀ ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
stderr:
# Failed test at /home/andres/src/postgresql/contrib/amcheck/t/001_verify_heapam.pl line 20.
# Failed test at /home/andres/src/postgresql/contrib/amcheck/t/001_verify_heapam.pl line 22.
# test logfile: /srv/dev/build/m/testrun/amcheck/001_verify_heapam/log/regress_log_001_verify_heapam
# Looks like you failed 2 tests of 275.

(test program exited with status code 2)
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――

which should make it a lot easier to find the log?

Greetings,

Andres Freund