initdb caching during tests

From: Andres Freund <andres(at)anarazel(dot)de>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: initdb caching during tests
Date: 2023-08-05 19:56:56
Message-ID: 20230805195656.wouaaf6qvrdo2cjt@awork3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

We have some issues with CI on macos and windows being too expensive (more on
that soon in a separate email), which reminded me of this thread (with
original title: [1])

I've attached a somewhat cleaned up version of the patch to cache initdb
across runs. The results are still fairly impressive in my opinion.

One thing I do not like, but don't have a good idea for how to improve, is
that there's a bunch of duplicated logic in pg_regress.c and Cluster.pm. I've
tried to move that into initdb.c itself, but that ends up pretty ugly, because
we need to be a lot more careful about checking whether options are compatible
etc. I've also thought about just putting this into a separate perl script,
but right now we still allow basic regression tests without perl being
available. So I concluded that for now just having the copies is the best
answer.

Times for running all tests under meson, on my workstation (20 cores / 40
threads):

cassert build -O2:

Before:
real 0m44.638s
user 7m58.780s
sys 2m48.773s

After:
real 0m38.938s
user 2m37.615s
sys 2m0.570s

cassert build -O0:

Before:
real 1m11.290s
user 13m9.817s
sys 2m54.946s

After:
real 1m2.959s
user 3m5.835s
sys 1m59.887s

non-cassert build:

Before:
real 0m34.579s
user 5m30.418s
sys 2m40.507s

After:
real 0m27.710s
user 2m20.644s
sys 1m55.770s

On CI this reduces the test times substantially:
Freebsd 8:51 -> 5:35
Debian w/ asan, autoconf 6:43 -> 4:55
Debian w/ alignmentsan, ubsan 4:02 -> 2:33
macos 5:07 -> 4:29
windows 10:21 -> 9:49

This is ignoring a bit of run-to-run variance, but the trend is obvious enough
that it's not worth worrying about that.

Greetings,

Andres Freund

[1] https://postgr.es/m/20220120021859.3zpsfqn4z7ob7afz%40alap3.anarazel.de

Attachment Content-Type Size
v3-0001-Use-template-initdb-in-tests.patch text/x-diff 10.4 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2023-08-05 20:25:39 ci: Improve macos startup using a cached macports installation
Previous Message Dean Rasheed 2023-08-05 19:50:01 Re: cataloguing NOT NULL constraints