Re: pg_upgade vs config

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> On 10/02/2016 12:54 PM, Tom Lane wrote:
>> [ digs more deeply ... ] Oh, weird: it looks like this succeeded in
>> every case except 9.6->HEAD upgrade. Did we break something recently?

> Yeah, my latest version of the test module (soon to hit githyb) also
> does a self upgrade, and these modules pass that on 9.5, whereas they
> fail on 9.6, as well as the 9.6->HEAD and HEAD self-tests failing. So
> indeed it looks like we've broken something.

I've now checked that simply doing LOAD 'hstore_plpython2.so' in a fresh
session fails, in both HEAD and 9.5, on both Linux and current macOS.
So it doesn't seem that we've broken anything since 9.5 --- it didn't
work before either. The seeming successes may have been due to chance,
i.e. pg_upgrade probing the libraries in an order that happened to work.
I see no evidence that get_loadable_libraries/check_loadable_libraries
are paying any attention to what order the libraries are checked in.

I've found that the Linux '-l:hstore.so' solution works on HPUX as well,
at least to the extent of being able to run LOAD. However, it doesn't
seem to be possible to make it work on macOS, which has a hard distinction
between "loadable modules" and "shared libraries". Our extensions are
the former, which means that e.g. hstore.so is not a shlib and the linker
will refuse to consider it as a linkable library. I tried converting
them to true shlibs, which is just a matter of changing "-bundle" to
"-dynamiclib" in the link command and dropping "-bundle_loader postgres"
because that's not accepted with "-dynamiclib". That works about 90%,
but I found that plpython failed its regression test for a very scary
reason: there's a "hash_search()" somewhere in libc on this platform
and the linker was preferentially resolving plpython's call to that
rather than to the one in Postgres. I don't think we want to risk
that sort of platform dependency :-(

On further reflection, though, this approach is probably a dead end
even without any platform concerns. The reason is that if we do
'LOAD foo' and the dynamic linker then pulls in 'bar.so', bar.so
will be present in the address space all right, but we will not have
checked for a compatible magic block, nor have called its _PG_init
function if any. In general, if calls into a module occur without
having called its _PG_init, it might misbehave very badly.

So now I'm thinking you're right, it'd be better to have some solution
whereby dfmgr.c knows about cross-module dependencies and loads the
dependencies first. Not sure how to approach that. The extension
"requires" mechanism is tantalizingly close to providing the data
we need, but dfmgr.c doesn't know about that, and there's no concept
of a reverse mapping from .so names to extensions anyway.

regards, tom lane