Re: Improving prep_buildtree used in VPATH builds

From: Greg Smith <greg(at)2ndquadrant(dot)com>
To: Gurjeet Singh <singh(dot)gurjeet(at)gmail(dot)com>
Cc: PGSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Improving prep_buildtree used in VPATH builds
Date: 2010-11-19 04:52:34
Message-ID: 4CE60292.7020908@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Gurjeet Singh wrote:
> Seems like it would improve performance in general, but more so under
> load conditions when you actually need it. I am not sure if -depth
> option is supported by all incarnations of 'find'.

Given the way directory writes are cached by the filesystem, I'm not
sure why the load at the time matters so much. If what you mean by that
is you're low on memory, that would make more sense to me.

Anyway, "-depth" is in POSIX as of 2001. It seems to be in all the
major SysV UNIX variants going back further then that, and of course
it's in GNU find. But it looks like BSD derived systems from before
that POSIX standard originally called this "-d" instead. So there's
some potential for this to not work on older systems; it works fine on
my test FreeBSD 7.2 system. Maybe someone who has access to some
ancient BSD-ish system can try this out? The simplest test case similar
to what the patch adds seems to be if this runs, returning
subdirectories in depth-first order before their parent:

$ find / -depth -type d -print

If that fails somewhere, it may turn out to require another configure
check just to determine whether you can use this configuration time
optimization. That's certainly possible to add to the patch if it got
committed and turns out to break one of the buildfarm members. It seems
to me like this whole thing may be a bit more work than it's worth,
given this is a fairly small and difficult to reproduce speedup in only
one stage of the build process. I'd think that if configure takes
longer than it has to because the system is heavily loaded, the amount
compilation time is going to suffer from that would always dwarf this
component of total build time. But if this was slow enough at some
point to motivate you to write a patch for it, maybe that assumption is
wrong.

--
Greg Smith 2ndQuadrant US greg(at)2ndQuadrant(dot)com Baltimore, MD
PostgreSQL Training, Services and Support www.2ndQuadrant.us

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Pavel Stehule 2010-11-19 06:40:57 Re: final patch - plpgsql: for-in-array
Previous Message Tom Lane 2010-11-19 04:38:14 Re: Latches with weak memory ordering (Re: max_wal_senders must die)