|From:||Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>|
|To:||Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>|
|Cc:||PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org>|
|Subject:||Re: PSA: we lack TAP test coverage on NetBSD and OpenBSD|
|Views:||Raw Message | Whole Thread | Download mbox|
> Although we've got a few NetBSD and OpenBSD buildfarm critters,
> none of them are doing --enable-tap-tests. If they were, we'd
> have noticed the pgbench regression tests falling over:
> I am, TBH, inclined to fix this by removing that test case rather
> than teaching it another spelling to accept. I think it's very
> hard to make the case that tests like this one are anything but
> a waste of developer and buildfarm time. When they are also a
> portability hazard, it's time to cut our losses. (I also note
> that this test has caused us problems before, cf 869aa40a2 and
I'd rather keep it by simply adding the "|unknown" alternative. 30 years
of programming have taught me that testing limit & error cases is useful,
although you never know when it will be proven so.
Client application coverage is currently abysmal, especially "psql"
despite the many script used for testing (39% of lines, 42% of
functions!), pgbench is under 90%. Generally we really need more tests,
not less. TAP tests are a good compromise because they are not always
run, and ISTM sometimes (i.e. you asked for it) is enough.
I agree that some tests can be useless, but I do not think that it applies
to this one. This test also checks that under a bad option pgbench stops
with an appropriate 1 exit status. Recently a patch updated the exit
status of pgbench in many cases to distinguish between different kind
errors, thus having non-regression in this area was shown to be a good
idea. Moreover, knowing that the exit status on getopt errors is
consistent across platform has value, and knowing that there is some
variability is not uninteresting.
|Next Message||Chris Travers||2019-01-17 09:50:56||Re: Proposal for Signal Detection Refactoring|
|Previous Message||Kyotaro HORIGUCHI||2019-01-17 09:28:48||Re: add_partial_path() may remove dominated path but still in use|