Re: make installcheck-world in a clean environment

On Mon, May 07, 2018 at 01:07:15PM -0400, Tom Lane wrote:
> Robert Haas <robertmhaas(at)gmail(dot)com> writes:
>> After thinking about this some more, I think the question here is
>> definitional. A first attempt at defining 'make installcheck' is to
>> say that it runs the tests from the build tree against the running
>> server. Certainly, we intend to use the SQL files that are present in
>> the build tree, not the ones that were present in the build tree where
>> the running server was built. But what about client programs that we
>> use to connect to the server? You're suggesting that we use the
>> pre-installed ones, but that is actually pretty problematic because
>> the ones we see as installed might correspond neither to the contents
>> of the build tree nor to the running server. Conceivably we could end
>> up having a mix of assets from three different places: (1) the running
>> server, (2) the build tree, (3) whatever is in our path at the moment.
>> That seems very confusing. So now I think it's probably right to
>> define 'make installcheck' as using the assets from the build tree to
>> test the running server. Under that definition, we're missing some
>> dependencies, but USE_INSTALLED_ASSETS isn't a thing we need.
>
> Nah, I disagree with this. To me, the purpose of "make installcheck"
> is to verify the correctness of an installation, which I take to include
> the client programs as well as the server. I think that "make
> installcheck" ought to use the installed version of any file that we
> actually install, and go to the build tree only for things we don't
> install (e.g. SQL test scripts).
>
> If the user has screwed up his PATH or other environmental aspects so that
> what he's testing isn't a single installation, that's his error, not
> something that "make installcheck" ought to work around. Indeed, maybe
> such aspects of his setup are intentional, and second-guessing them would
> completely defeat his purpose. In any case, if you want to test the
> build-tree assets, that's what "make check" is for.

I agree with Tom's position, and this is the behavior that Postgres is
using for ages for check and installcheck. If there are no objections,
I'll mark the patch as rejected and move on to other things.
--
Michael