faster testing with symlink installs

From: Peter Eisentraut <peter(dot)eisentraut(at)2ndquadrant(dot)com>
To: pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: faster testing with symlink installs
Date: 2018-03-01 02:34:06
Message-ID: 5d14d20c-b1f6-69c8-7465-df8adb424aea@2ndquadrant.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I'm proposing a way to make test runs a bit faster.

A fair amount of time is taken by creating the test installation. Not
huge compared to the whole test run, but still long enough to get
boring. The idea (that I stole from somewhere else) is that we make the
installation as symlinks pointing back to the source tree. So after the
first run, the installation is automatically up to date after each
build. Then we can skip the whole temp install run if we find that we
already have one (unless you change the set of installed files in a
major way, which is very rate). This also helps if you use make
installcheck: You just need to rebuild and restart the server, skipping
the make install run.

So that's what the patch does: make install INSTALL_SYMLINKS=1 installs
files by making relative symlinks. make check TEMP_INSTALL_SYMLINKS=1
creates the temporary installation using symlinks and skips the install
step if a tmp_install already exists. It requires the "ln" program from
GNU coreutils. So it would be optional.

Except ... this doesn't actually work. find_my_exec() resolves symlinks
to find the actual program installation, and so for example the
installed initdb will look for postgres in src/bin/initdb/. I would
like to revert this behavior, because it seems to do more harm than
good. The original commit message indicates that this would allow
symlinking a program to somewhere outside of the installation tree and
still allow it to work and find its friends. But that could just as
well be done with a shell script.

Reverting this behavior would also allow Homebrew-like installations to
work better. The actual installation is in a versioned directory like
/usr/local/Cellar/postgresql/10.1/... but symlinked to
/usr/local/opt/postgresql/. But because of the symlink resolution,
calling /usr/local/opt/postgresql/bin/pg_config returns paths under
Cellar, which then causes breakage of software built against that path
on postgresql upgrades the change the version number.

Thoughts?

--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Attachment Content-Type Size
0001-Remove-duplicate-installation-of-some-header-files.patch text/plain 2.3 KB
0002-Don-t-resolve-symlinks-in-find_my_exec.patch text/plain 4.5 KB
0003-Add-TEMP_INSTALL_SYMLINKS-mode.patch text/plain 2.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2018-03-01 02:43:44 Re: Support for ECDSA & ed25519 digital signatures in pgcrypto?
Previous Message Amit Langote 2018-03-01 02:27:04 Re: [HACKERS] path toward faster partition pruning