JIT inlining crashes on non-exported extension functions

Hi hackers,

With JIT inlining enabled but optimization not, queries using some
contrib functions kill the session at code emission:

SET jit = on;
SET jit_above_cost = 0;
SET jit_inline_above_cost = 0;
SET jit_optimize_above_cost = -1;
SELECT page_checksum(get_raw_page('t', 0), 0);
FATAL: fatal llvm error: Program used external function
'get_page_from_raw' which could not be resolved!

Since 089480c0770 extension libraries are built with
-fvisibility=hidden, so get_page_from_raw(), ltree's compare_subnode()
and pgcrypto's mbuf_create_from_data() aren't exported from their
libraries. The inliner doesn't know that: when it imports
page_checksum_1_9() it leaves get_page_from_raw() as an external call,
expecting it to be resolved at emission time, and it isn't.

When jit_optimize_above_cost is reached too, O3 often inlines the helper
as well and the reference goes away. With the default settings both
thresholds are 500000, which is probably why nobody has reported this.
But it only takes jit_inline_above_cost being reached without
jit_optimize_above_cost, both of which any user can set, and nothing
guarantees that O3 inlines the helper either. jit is off by default on
master since 7f8c88c2b87, but not on the back branches.

I found this while moving our test hosts from LLVM 19 to 21. It isn't
specific to one LLVM version: the same contrib tests fail on Debian 13
with 19.1.7 and 21.1.8, on x86_64 and aarch64, when check-world is run
with

PG_TEST_INITDB_EXTRA_OPTS='-c jit=on -c jit_above_cost=0
-c jit_inline_above_cost=0 -c jit_optimize_above_cost=-1'

pageinspect: page, checksum, oldextversions
ltree: ltree
pgcrypto: pgp-info (x86_64 only)

The attached patch makes function_inlinable() refuse a function that
refers to an external function or variable that can't be resolved, using
the same lookup llvm_resolve_symbol() does at emission time. Exported
functions are still inlined as before (int4pl into a plain expression,
citext_eq, hstore_exists); ltree's ltxtq_exec is now left as a call
instead of crashing. The cost is one dlsym() per external symbol
referenced by an inlining candidate; I haven't measured it separately,
but in the 19 vs 21 runs below the inlining phase didn't get slower.

Compiling the bitcode with -fvisibility=hidden and checking symbol
visibility in the inliner instead wouldn't help bitcode already
installed by extensions, nor cover libraries that restrict their exports
some other way, so I went with the lookup.

The patch adds a JIT case to pageinspect's page test, which fails
without the fix even on a plain "make check", since it sets the jit GUCs
itself. Without LLVM it passes trivially. It relies on
get_page_from_raw() staying unexported, and only bites on builds that
install extension bitcode, which today means autoconf builds; meson will
too once CF 5671 (LLVM bitcode for meson builds) is in.

With the patch, check-world has no failures with inlining forced (above)
or with inlining and optimization forced (jit_optimize_above_cost = 0),
on LLVM 19 and 21, x86_64 and aarch64. The jit=off and
JIT-without-inlining modes I ran only on unpatched master, where 19 and
21 gave identical results; the patch only touches function_inlinable(),
which those modes don't reach.

I'd suggest backpatching to 16, where hidden visibility arrived. It
applies to 16 through 19 with offsets.

One ordering problem: the new test is the first in the tree that turns
JIT on, so on CI's AddressSanitizer job (Linux - Meson 64-bit) it hits
the unrelated ASAN + JIT crash Henson Choi is fixing in [1], and the
backend dies of SIGILL there. So the test can only go in together with
or after that fix. If that's going to take a while, I'd suggest that we
split the test out and apply it as a follow-up after [1] lands.

best.

-greg

[1] https://postgr.es/m/CAAAe_zA49cHXJ7J4hmeuSa75s%2BeSgCOnK_9Uvq6wUEJ2Bgvdfg%40mail.gmail.com