[PATCH] Skip llvm bytecode generation if LLVM is missing

From: Craig Ringer <craig(at)2ndquadrant(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Andres Freund <andres(at)anarazel(dot)de>, Devrim Gündüz <devrim(at)gunduz(dot)org>
Subject: [PATCH] Skip llvm bytecode generation if LLVM is missing
Date: 2020-03-11 03:25:28
Message-ID: CAMsr+YFru8nqv9zAay8jq3YhL0HOuyAEUNJ9EZsLd3rnJ5=OVQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Short version: Currently if the server is built with --with-llvm the
-devel packages must depend on clang for PGXS to work, even though
llvm is otherwise optional. This is a particular problem on older
platforms where 3rd party LLVM may be required to build the server's
llvmjit support. Work around by skipping the default .bc generation if
no clang is found by PGXS, as if $(with_llvm) was false.

Detail:

If PostgreSQL is configured with --with--lvm it writes with_llvm=yes
into Makefile.global via AC_SUBST, along with the CLANG path and the
path to the LLVM_TOOLSET if supplied. PGXS sets up a %.bc dependency
for OBJS if it detects that the server was compiled with llvm support.

If clang is not found at PGXS extension build-time the extension build
will then fail, despite the user not having installed the
postgresql11-llvmjit (or whatever) package and the extension not
declaring any explicit LLVM dependency or requirement.

Andres and others went to a great deal of effort to make it possible
to separate PostgreSQL's LLVM support, so a server built with LLVM
support doesn't actually have a runtime dependency on llvm unless the
llvmjit module is loaded. This allows packagers to separate it out and
avoid the need to declare an llvm dependency on the main server
package.

I've found that this falls down for -devel packages like those the
PGDG Yum team produces. The problem arises when a container or VM is
used to build the server and its Makefile.global etc. Then the -devel
package containing Makefile.global and other PGXS bits are installed
on a new machine that does not have llvm's clang. PGXS builds will
fail when they attempt to generate bytecode, since they expect clang
to be present at the path baked in to Makefile.global - but it's
absent.

I propose that per the attached patch PGXS should simply skip adding
the automatic dependency for .bc files if clang cannot be found.
Extensions may still choose to explicitly declare the rule in their
own Makefile if they want to force bitcode generation.

If we want to get fancier about it, we could instead split the llvm
support out from Makefile.global into a Makefile.llvm or similar,
which is then conditionally included by Makefile.global if it exists.
Makefile.llvm would be packaged in a new postgresqlXX-llvmjit-devel
package since distros get uppity if you put makefile fragments in
runtime packages. This package would depend on llvm-toolset and clang.
If you install postgresqlXX-devel but not postgresqlXX-llvmjit-devel
you don't get bitcode and don't need clang. If you install
postgresqlXX-llvmjit-devel, the same clang as we built the server with
is declared as a dependency and Makefile.llvm is included, so it all
works.

But I don't think it's worth the hassle and I'd rather just skip
automatic bitcode generation if we don't find clang.

See also yum packagers report at
https://www.postgresql.org/message-id/CAMsr+YHokx0rWLV561z3=gAi6CM4YJekgCLkqmDwQSTEjVYhuw@mail.gmail.com
.

--
Craig Ringer http://www.2ndQuadrant.com/
2ndQuadrant - PostgreSQL Solutions for the Enterprise

Attachment Content-Type Size
Makefile-llvm-toolset.patch text/x-patch 952 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2020-03-11 03:26:20 Re: [Patch] pg_rewind: options to use restore_command from recovery.conf or command line
Previous Message Thomas Munro 2020-03-11 03:13:17 Re: Collation versioning