Re: pg_regress: Treat child process failure as test failure

From: Andres Freund <andres(at)anarazel(dot)de>
To: Daniel Gustafsson <daniel(at)yesql(dot)se>
Cc: Postgres hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: pg_regress: Treat child process failure as test failure
Date: 2022-11-26 20:55:59
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


On 2022-11-26 21:11:39 +0100, Daniel Gustafsson wrote:
> In the thread about TAP format out in pg_regress, Andres pointed out [0] that
> we allow a test to pass even if the test child process failed. While its
> probably pretty rare to have a test pass if the process failed, this brings a
> risk for false positives (and it seems questionable that any regress test will
> have a child process failing as part of its intended run).

> The attached makes child failures an error condition for the test as a belts
> and suspenders type check. Thoughts?

I wonder if it's the right thing to treat a failed psql that's then also
ignored as "failed (ignored)". Perhaps it'd be better to move the statuses[i]
!= 0 check to before the if (differ)?

> - if (differ)
> + if (differ || statuses[i] != 0)
> {
> bool ignore = false;
> _stringlist *sl;
> @@ -1815,7 +1815,7 @@ run_single_test(const char *test, test_start_function startfunc,
> differ |= newdiff;
> }
> - if (differ)
> + if (differ || exit_status != 0)
> {
> status(_("FAILED"));
> fail_count++;

It certainly is a bit confusing that we print a psql failure separately from
the if "FAILED" vs "ok" bit.


Andres Freund

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2022-11-26 21:05:25 Re: MSVC vs Perl
Previous Message Daniel Gustafsson 2022-11-26 20:35:45 Re: TAP output format in pg_regress