Re: JIT breaks PostGIS

Re: Andres Freund 2018-07-25 <20180725191143(dot)sietxlbfehv245hb(at)alap3(dot)anarazel(dot)de>
> I haven't investigated the details here. It certainly would be possible
> to have the _PG_init() of postgis's so force JIT to be off, and emit a
> warning.

Isn't that too late, if postgis.so gets loaded by a query that is in
to process of being jit'ed?

> There's no way to just force JIT off whenever anything involving postgis
> is present in the query. Not installing the .bc files will prevent
> inlining, but I don't think that's sufficient to prevent the problem in
> all cases.

Ok.

Different question, the other way round, is there a way to know that
all files needed to inline a query/extension are there? How does the
JIT machinery determine it can (try to) compile things? (That's
something extension packages might want to test for.)

> > , or should we look into getting a new version to distribute along
> > with it on apt.postgresql.org? I'd rather like to avoid having to ship
> > a compiler...
>
> Well, you don't need to ship the compiler (clang), strictly
> speaking. But yea.

We need clang to compile PostgreSQL and extensions, so it needs to
come from somewhere. We could pull it from (stretch-)backports, but
having to enable backports for really every build doesn't look
appealing. (At the moment it's about two dozen out of ~100 packages
that needs backports at build time, and half of that is python
packages that are dependencies of pgadmin4 only.)

> > > It also may happen that a patch for LLVM can be applied to LLVM4 build in
> > > Debian and brought in as an update, but such a plan should not be a default
> > > one.
> >
> > That's actually a plan I like very much. Hopefully the patch is small
> > and backpatchable.
>
> It's not entirely trivial, and I suspect there's independent changes
> making it not apply cleanly:
> https://github.com/llvm-mirror/llvm/commit/ab3dba86f951a1bdfe01d3102e2850e607af791a

The svn link for that is https://llvm.org/viewvc/llvm-project?view=revision&revision=302589

I tried to apply the patch to llvm-toolchain-4.0_4.0.1-10~deb9u2.
There are no rejects, but two of the patched files do not even exist.

patching file include/llvm/ExecutionEngine/Orc/CompileOnDemandLayer.h
patching file include/llvm/ExecutionEngine/Orc/OrcRemoteTargetClient.h
Hunk #1 succeeded at 143 (offset -1 lines).
Hunk #2 succeeded at 319 (offset -1 lines).
Hunk #3 succeeded at 330 (offset -1 lines).
Hunk #4 succeeded at 387 (offset -1 lines).
can't find file to patch at input line 171
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h b/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
|index babcc7f26aab..5b3426afe584 100644
|--- a/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
|+++ b/include/llvm/ExecutionEngine/Orc/RTDyldObjectLinkingLayer.h
--------------------------
File to patch:
Skip this patch? [y]
Skipping patch.
1 out of 1 hunk ignored
patching file include/llvm/ExecutionEngine/RTDyldMemoryManager.h
patching file include/llvm/ExecutionEngine/RuntimeDyld.h
patching file lib/ExecutionEngine/Orc/OrcMCJITReplacement.h
patching file lib/ExecutionEngine/RuntimeDyld/RTDyldMemoryManager.cpp
patching file lib/ExecutionEngine/RuntimeDyld/RuntimeDyld.cpp
patching file lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.cpp
patching file lib/ExecutionEngine/RuntimeDyld/RuntimeDyldELF.h
Hunk #1 succeeded at 144 with fuzz 2 (offset -8 lines).
Hunk #2 succeeded at 167 (offset -12 lines).
patching file lib/ExecutionEngine/RuntimeDyld/RuntimeDyldImpl.h
Hunk #1 succeeded at 504 (offset -11 lines).
patching file lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFI386.h
patching file lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFThumb.h
patching file lib/ExecutionEngine/RuntimeDyld/Targets/RuntimeDyldCOFFX86_64.h
patching file tools/lli/RemoteJITUtils.h
patching file tools/llvm-rtdyld/llvm-rtdyld.cpp
patching file unittests/ExecutionEngine/Orc/ObjectTransformLayerTest.cpp
can't find file to patch at input line 429
Perhaps you used the wrong -p or --strip option?
The text leading up to this was:
--------------------------
|diff --git a/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp b/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
|index de99c022fb9d..c13a75a5cbfe 100644
|--- a/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
|+++ b/unittests/ExecutionEngine/Orc/RTDyldObjectLinkingLayerTest.cpp
--------------------------
File to patch:
Skip this patch? [y]
Skipping patch.
5 out of 5 hunks ignored

Christoph