Re: improving speed of make check-world

On Tue, Apr 28, 2015 at 1:46 AM, Jeff Janes <jeff(dot)janes(at)gmail(dot)com> wrote:
> On Sat, Apr 25, 2015 at 7:23 AM, Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
> wrote:
>>
>> On Sat, Apr 25, 2015 at 7:59 AM, Peter Eisentraut <peter_e(at)gmx(dot)net> wrote:
>> > On 4/23/15 1:22 PM, Jeff Janes wrote:
>> >> Something about this commit (dcae5faccab64776376d354d) broke "make
>> >> check" in parallel conditions when started from a clean directory. It
>> >> fails with a different error each time, one example:
>> >>
>> >> make -j4 check > /dev/null
>> >>
>> >> In file included from gram.y:14515:
>> >> scan.c: In function 'yy_try_NUL_trans':
>> >> scan.c:10307: warning: unused variable 'yyg'
>> >> /usr/bin/ld: tab-complete.o: No such file: No such file or directory
>> >> collect2: ld returned 1 exit status
>> >> make[3]: *** [psql] Error 1
>> >> make[2]: *** [all-psql-recurse] Error 2
>> >> make[2]: *** Waiting for unfinished jobs....
>> >> make[1]: *** [all-bin-recurse] Error 2
>> >> make: *** [all-src-recurse] Error 2
>> >> make: *** Waiting for unfinished jobs....
>> >
>> > I think the problem is that "check" depends on "all", but now also
>> > depends on temp-install, which in turn runs install and all. With a
>> > sufficient amount of parallelism, you end up running two "all"s on top
>> > of each other.
>> >
>> > It seems this can be fixed by removing the check: all dependency. Try
>> > removing that in the top-level GNUmakefile.in and see if the problem
>> > goes away. For completeness, we should then also remove it in the other
>> > makefiles.
>>
>> Yep. I spent some time yesterday and today poking at that, but I
>> clearly missed that dependency.. Attached is a patch fixing the
>> problem. I tested check and check-world with some parallel jobs and
>> both worked. In the case of check, the amount of logs is very reduced
>> because all the install process is done by temp-install which
>> redirects everything into tmp_install/log/install.log.
>
>
> This change fixed the problem for me.
>
> It also made this age-old compiler warning go away:
>
> In file included from gram.y:14515:
> scan.c: In function 'yy_try_NUL_trans':
> scan.c:10307: warning: unused variable 'yyg'
>
> I guess by redirecting it into the log file you indicated, but is that a
> good idea to redirect stderr?

I am sure that Peter did that on purpose, both approaches having
advantages and disadvantages. Personally I don't mind looking at the
install log file in tmp_install to see the state of the installation,
but it is true that this change is a bit disturbing regarding the fact
that everything was directly outputted to stderr and stdout for many
years.
--
Michael