Re: Speed up build on Windows by generating symbol definition in batch

On Sat, Mar 30, 2019 at 03:42:39PM +0900, Peifeng Qiu wrote:
> When I watched the whole build process with a task manager, I discovered
> that a lot of time was spent on generating export symbol definitions,
> without consuming much CPU or IO.
> The script that doing this is src/tools/msvc/gendef.pl, it enumerates the
> whole directory for ".obj" files and call dumpbin utility to generate
> ".sym" files one by one like this:
>
> dumpbin /symbols /out:a.sym a.obj >NUL
>
> Actually the dumpbin utility accepts a wildcard file name, so we can
> generate the export symbols of all ".obj" files in batch.
>
> dumpbin /symbols /out:all.sym *.obj >NUL
>
> This will avoid wasting time by creating and destroying dumpbin process
> repeatedly and can speed up the build process considerably.
> I've tested on my 4-core 8-thread Intel i7 CPU. I've set MSBFLAGS=/m to
> ensure it can utilize all CPU cores.
> Building without this patch takes about 370 seconds. Building with this
> patch takes about 200 seconds. That's almost 2x speed up.

I, too, get a strong improvement, from 201s to 149s. I can confirm it yields
identical *.def files. Thanks for identifying this improvement.

> - my ($objfile, $symfile) = @_;
> - my ($symvol, $symdirs, $symbase) = splitpath($symfile);
> - my $tmpfile = catpath($symvol, $symdirs, "symbols.out");

You removed the last use of File::Spec::Functions, so remove its "use"
statement.

> - system("dumpbin /symbols /out:$tmpfile $_ >NUL")
> - && die "Could not call dumpbin";

This error handling was crude, but don't replace it with zero error handling.

> - rename($tmpfile, $symfile);

Keep the use of a temporary file, too.

> +system("dumpbin /symbols /out:$symfile $ARGV[0]/*obj >NUL");

That should be *.obj, not *obj.