Reducing buildfarm disk usage: remove temp installs when done

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org
Subject: Reducing buildfarm disk usage: remove temp installs when done
Date: 2015-01-18 22:48:11
Message-ID: 18617.1421621291@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: buildfarm-members pgsql-hackers

One of the biggest causes of buildfarm run failures is "out of disk
space". That's not just because people are running buildfarm critters
on small slow machines; it's because "make check-world" is an enormous
space hog. Some numbers from current HEAD:

clean source tree: 120MB
built source tree: 400MB
tree after make check-world: 3GB

(This is excluding ~250MB for one's git repo.)

The reason for all the bloat is the temporary install trees that we
create, which tend to eat up about 100MB apiece, and there are dozens
of them (eg, one per testable contrib module). Those don't get removed
until the end of the test run, so the usage is cumulative.

The attached proposed patch removes each temp install tree as soon as
we're done with it, in the normal case where no error was detected.
This brings the peak space usage down from ~3GB to ~750MB.

To make things better in the buildfarm, we'd have to back-patch this into
all active branches, but I don't see any big problem with doing so.

Any objections?

regards, tom lane

Attachment Content-Type Size
clean-up-temp-installs-immediately.patch text/x-diff 844 bytes

Responses

Browse buildfarm-members by date

  From Date Subject
Next Message Michael Paquier 2015-01-18 23:48:26 Re: Reducing buildfarm disk usage: remove temp installs when done
Previous Message Marti Raudsepp 2014-09-22 09:59:25 Re: [Pgbuildfarm-members] Submission failures: 500 read timeout

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2015-01-18 23:48:26 Re: Reducing buildfarm disk usage: remove temp installs when done
Previous Message Tom Lane 2015-01-18 21:09:29 Re: Re: Better way of dealing with pgstat wait timeout during buildfarm runs?