Re: Reducing buildfarm disk usage: remove temp installs when done

On 2015-01-18 17:48:11 -0500, Tom Lane wrote:
> One of the biggest causes of buildfarm run failures is "out of disk
> space". That's not just because people are running buildfarm critters
> on small slow machines; it's because "make check-world" is an enormous
> space hog. Some numbers from current HEAD:
>
> clean source tree: 120MB
> built source tree: 400MB
> tree after make check-world: 3GB
>
> (This is excluding ~250MB for one's git repo.)
>
> The reason for all the bloat is the temporary install trees that we
> create, which tend to eat up about 100MB apiece, and there are dozens
> of them (eg, one per testable contrib module). Those don't get removed
> until the end of the test run, so the usage is cumulative.
>
> The attached proposed patch removes each temp install tree as soon as
> we're done with it, in the normal case where no error was detected.
> This brings the peak space usage down from ~3GB to ~750MB.

I was wondering before if we couldn't always do the the temp
installation into $top_builddir/tmp_install or something like it. With
an additional small ugly hacking ontop we could even avoid reinstalling
for every target in check-world.

Greetings,

Andres Freund

--
Andres Freund http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services