Re: pg_upgrade automatic testing

On 11/27/2011 06:17 PM, Tom Lane wrote:
> Peter Eisentraut<peter_e(at)gmx(dot)net> writes:
>> I've committed it now, and some buildfarm members are failing with lack
>> of shared memory, semaphores, or disk space. Don't know what to do with
>> that or why so many are failing like that. We could create a way to
>> omit the test if it becomes a problem.
> I believe the issue is that those BF members have kernel settings that
> only support running one postmaster at a time. The way you've got this
> set up, it launches a new private postmaster during a make installcheck;
> which is not only problematic from a resource consumption standpoint,
> but seems to me to violate the spirit of make installcheck, because
> what it's testing is not the installed postmaster but a local instance.
>
> Can you confine the test to only occur in "make check" mode, not "make
> installcheck", please?
>
>

Contrib tests are only run by the buildfarm in installcheck mode, so
that will probably turn the tests off for the buildfarm, so +1 on that
:-) I think these tests are probably somewhat ill-conceived. Note also
that shell scripts are not portable, and so these tests would not be
able to run on MSVC buildfarm members, even if they had been enabled in
the MSVC regression driver, which they have not. If we need a regression
driver script it needs to be written in Perl.

Also note that the test as written is likely to add significantly to
buildfarm run times, as it will run the entire base regression suite
again, possibly several times.

Finally, I think that this is probably not what we really need. I have
already started work (as I mentioned some weeks ago) on having the
buildfarm stash away a successful build and data directory, to be used
later in cross-version upgrade testing, which seems to me much more what
we need from something like the buildfarm. Maybe we could discuss how to
run something like that.

And yes, some buildfarm members run on fairly scarce machine resources,
of memory, CPU time and disk space, and we need not to increase what our
tests use without due notice and care.

cheers

andrew