Re: LLVM 16 (opaque pointers)

From: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Cc: Fabien COELHO <coelho(at)cri(dot)ensmp(dot)fr>, Ronan Dunklau <ronan(dot)dunklau(at)aiven(dot)io>, Andres Freund <andres(at)anarazel(dot)de>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>
Subject: Re: LLVM 16 (opaque pointers)
Date: 2023-09-20 20:22:20
Message-ID: CA+hUKG+UQq_qZWuAd8Mv=RpL4sACEb=4UNgNL2--=xZHixVspg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Belated thanks Dmitry, Ronan, Andres for your feedback. Here's a new
version, also including Dmitry's patch for 17 which it is now also
time to push. It required a bit more trivial #if magic to be
conditional, as Dmitry already mentioned. I just noticed that Dmitry
had the LLVMPassBuilderOptionsSetInlinerThreshold() function added to
LLVM 17's C API for this patch. Thanks! (Better than putting stuff
in llvmjit_wrap.c, if you can get it upstreamed in time.)

I thought I needed to block users from building with too-old clang and
too-new LLVM, but I haven't managed to find a combination that
actually breaks anything. I wouldn't recommend it, but for example
clang 10 bitcode seems to be inlinable without problems by LLVM 16 on
my system (I didn't use an assert build of LLVM though). I think that
could be a separate adjustment if we learn that we need to enforce or
document a constraint there.

So far I've tested LLVM versions 10, 15, 16, 17, 18 (= their main
branch) against PostgreSQL versions 14, 15, 16. I've attached the
versions that apply to master and 16, and pushed back-patches to 14
and 15 to public branches if anyone's interested[1]. Back-patching
further seems a bit harder. I'm quite willing to do it, but ... do we
actually need to, ie does anyone really *require* old PostgreSQL
release branches to work with new LLVM?

(I'll start a separate thread about the related question of when we
get to drop support for old LLVMs.)

One point from an earlier email:

On Sat, Aug 12, 2023 at 6:09 AM Andres Freund <andres(at)anarazel(dot)de> wrote:
> > AttributeTemplate(PG_FUNCTION_ARGS)
> > {
> > + PGFunction fp PG_USED_FOR_ASSERTS_ONLY;
> > +
> > + fp = &AttributeTemplate;

> Other parts of the file do this by putting the functions into
> referenced_functions[], i'd copy that here and below.

Actually here I just wanted to assert that the 3 template functions
match certain function pointer types. To restate what these functions
are about: in the JIT code I need the function type, but we have only
the function pointer type, and it is now impossible to go from a
function pointer type to a function type, so I needed to define some
example functions with the right prototype (well, one of them existed
already but I needed more), and then I wanted to assert that they are
assignable to the appropriate function pointer types. Does that make
sense?

In this version I changed it to what I hope is a more obvious/explicit
expression of that goal:

+ AssertVariableIsOfType(&ExecEvalSubroutineTemplate,
+ ExecEvalSubroutine);

[1] https://github.com/macdice/postgres/tree/llvm16-14 and -15

Attachment Content-Type Size
v2-0001-jit-Support-opaque-pointers-in-LLVM-16.patch text/x-patch 59.3 KB
v2-0002-jit-Changes-for-LLVM-17.patch text/x-patch 3.2 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Nathan Bossart 2023-09-20 20:29:22 Re: Improvements in pg_dump/pg_restore toc format and performances
Previous Message Robert Haas 2023-09-20 20:20:08 Re: New WAL record to detect the checkpoint redo location