TRANSFORM modules vs. AIX

(was Re: [COMMITTERS] pgsql: Add transforms feature)

On Tue, May 05, 2015 at 10:24:03PM -0400, Peter Eisentraut wrote:
> Any dlopenable module will have undefined symbols, namely those that it
> uses to call back into the executable that loads the module. The only
> way it can know to not complain about those symbols is if we tell it
> about that executable at build time. Our current shared library build
> setup makes reference to the postgres executable only on three
> platforms: Windows, OS X, AIX. All other platforms necessarily accept
> all undefined symbols at build time. We already know that it can also
> be made to work on Windows and OS X. I expect that we might need some
> tweaking on AIX, but I'm quite sure it can be made to work there also,

We can address AIX almost exactly like we address Windows, attached.
Relocating an AIX installation will then require adding $(pkglibdir) to
LD_LIBRARY_PATH. Windows, too, must need the reference from ltree_plpython to
plpython; I'll add it separately. (It is for PLyUnicode_FromStringAndSize(),
which we omit under Python 2. The buildfarm's Windows+GCC+Python animal,
frogmouth, uses Python 2.)

contrib/hstore_plperl test coverage revealed a PL/Perl bug found in all
supported branches. Any import of a native-code module crashes:

create language plperlu;
CREATE FUNCTION test1none() RETURNS int LANGUAGE plperlu
AS $$ use IO; return 0; $$;

"man perlaix" gives the essential clue:

Note that starting from Perl 5.7.2 (and consequently 5.8.0) and AIX
4.3 or newer Perl uses the AIX native dynamic loading interface in
the so called runtime linking mode instead of the emulated interface
that was used in Perl releases 5.6.1 and earlier or, for AIX releases
4.2 and earlier. This change does break backward compatibility with
compiled modules from earlier perl releases. The change was made to
make Perl more compliant with other applications like Apache/mod_perl
which are using the AIX native interface. This change also enables
the use of C++ code with static constructors and destructors in perl
extensions, which was not possible using the emulated interface.

PostgreSQL uses AIX default linking, not AIX runtime linking. Perl module
shared objects (e.g. IO.so) contain undefined symbols and rely on the runtime
linker to resolve them. In an executable free from the runtime linker, those
symbols just remain NULL. I am inclined to back-patch the following:

--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -90,3 +90,3 @@ ifeq ($(PORTNAME), aix)
postgres: $(POSTGRES_IMP)
- $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(call expand_subsys,$(OBJS)) -Wl,-bE:$(top_builddir)/src/backend/$(POSTGRES_IMP) $(LIBS) -o $@
+ $(CC) $(CFLAGS) $(LDFLAGS) $(LDFLAGS_EX) $(call expand_subsys,$(OBJS)) -Wl,-bE:$(top_builddir)/src/backend/$(POSTGRES_IMP) $(LIBS) -Wl,-brtllib -o $@

That allows the runtime linker to process modules dlopen()'ed within the
postgres executable. To my knowledge, it changes nothing else.

At some point, we might switch from AIX default linking to runtime linking.
(Concretely, that entails building modules with -Wl,-G and the postgres
executable with -Wl,-brtl.) Runtime linking treats undefined symbols and
duplicate symbols much more like other modern systems treat them. I don't
personally wish to rock this boat to that degree. "ld -lfoo" ignores foo.so,
but "ld -brtl -lfoo" prefers foo.so over foo.a. Therefore, such a change
wouldn't be back-patch material. Details on AIX shared library linking:

http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.genprogc/shared_object_runtime_linking.htm
http://www-01.ibm.com/support/knowledgecenter/ssw_aix_71/com.ibm.aix.cmds3/ld.htm