Re: TAP tests - installcheck vs check

On 04/25/2017 10:45 AM, Robert Haas wrote:
> On Sun, Apr 23, 2017 at 10:57 PM, Andrew Dunstan
> <andrew(dot)dunstan(at)2ndquadrant(dot)com> wrote:
>> On 04/23/2017 10:33 PM, Tom Lane wrote:
>>> Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com> writes:
>>>> AFAICT, unlike the pg_regress checks, which in the installcheck case run
>>>> against a running instance of postgres, for TAP tests the only
>>>> difference is that that for the check case a temp install is done,
>>>> possibly with some extra contrib modules. Is that correct? If is is, why
>>>> aren't we providing an installcheck target for tests like recover. In at
>>>> least one case (buildfarmn jacana) installs are quite expensive (2 or 3
>>>> minutes) and if they are pointless as seems to be the case here why
>>>> can't we just avoid them?
>>> A lot of those test cases involve setting non-default configuration
>>> parameters and/or stopping/starting the postmaster. So I can't see how
>>> we would run them against a pre-existing live cluster, which is the usual
>>> meaning of "make installcheck".
>>>
>>> I think what you're imagining is skipping redundant builds of the
>>> "tmp_install" tree by using an installation tree with a temporary $PGDATA
>>> directory. That seems like a fine idea, but we need another word for it.
>> That's actually the current meaning of installcheck w.r.t. TAP. See
>> Makefile.global.in. It's what we run (mostly) in the buildfarm for the
>> bin tests.
>>
>> I agree entirely that it's confusing as heck. +1 for inventing a new name.
> Yeah. I would have expected installcheck to just skip any tests that
> don't make sense against an already-installed cluster. I would not
> expect it to run those tests against some cluster other than the
> installed cluster. That seems super-weird.
>

I'm in the process of moving all the buildfarm tests to use check
instead of installcheck, but in such a way that it doesn't constantly
generate redundant installs.

When I'm done testing that and we have a new buildfarm client release
and we get all those buildfarm owners updated (there are 13 animals not
controlled by me running TAP tests) I think we should just rename
installcheck. for TAP.

cheers

andrew

--

Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services