|From:||Andres Freund <andres(at)anarazel(dot)de>|
|To:||Daniel Gustafsson <daniel(at)yesql(dot)se>|
|Cc:||Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>|
|Subject:||Re: pg_regress: Treat child process failure as test failure|
|Views:||Raw Message | Whole Thread | Download mbox | Resend email|
On 2022-11-26 21:11:39 +0100, Daniel Gustafsson wrote:
> In the thread about TAP format out in pg_regress, Andres pointed out  that
> we allow a test to pass even if the test child process failed. While its
> probably pretty rare to have a test pass if the process failed, this brings a
> risk for false positives (and it seems questionable that any regress test will
> have a child process failing as part of its intended run).
> The attached makes child failures an error condition for the test as a belts
> and suspenders type check. Thoughts?
I wonder if it's the right thing to treat a failed psql that's then also
ignored as "failed (ignored)". Perhaps it'd be better to move the statuses[i]
!= 0 check to before the if (differ)?
> - if (differ)
> + if (differ || statuses[i] != 0)
> bool ignore = false;
> _stringlist *sl;
> @@ -1815,7 +1815,7 @@ run_single_test(const char *test, test_start_function startfunc,
> differ |= newdiff;
> - if (differ)
> + if (differ || exit_status != 0)
It certainly is a bit confusing that we print a psql failure separately from
the if "FAILED" vs "ok" bit.
|Next Message||Andres Freund||2022-11-26 21:05:25||Re: MSVC vs Perl|
|Previous Message||Daniel Gustafsson||2022-11-26 20:35:45||Re: TAP output format in pg_regress|