Re: dynloader.h missing in prebuilt package for Windows?

On Fri, Dec 4, 2015 at 07:09:03PM -0500, Chapman Flack wrote:
> On 12/04/15 12:56, Bruce Momjian wrote:
> >
> > OK, good logical reason to install dynloader.h on Windows.
>
> Ah! Thanks. I was starting to wonder whether I had said something wrong
> and been sent to the bit bucket within my first two -hackers posts. :)

No, sometimes people just don't know how to respond, particuarly related
to technology we don't use regularly.

> > What do we need to do to close this item? What versions of Windows
> > installers are missing dynloader.h? Mingw? MSVC? EDB's? OpenSCG?
> > Postgres Pro (Russian)?
>
> I am too new around here to be able to supply good answers myself to
> those questions. I keep learning though. For example, I have just
> learned that OpenSCG and Postgres Pro (Russian) are things, and (by
> inference) that they run on Windows. :)

Yes, there are several binary Postgres distributions. However, in
researching, I think there is a central way to fix them all.

> In working on PL/Java, as a non-Windows dev myself, I have been
> blessed to find Ken Olson willing to build my WIP branches in MSVC
> and tell me what I've busted. I think he's building against an EDB
> installation of PostgreSQL, so that's the combination I am least
> ignorant about. I know that mingw has also been used, but I haven't
> yet made a connection with someone who can tell me what I'm breaking
> for that build....

OK, that sounds good. I assume he is using MSVC to build PL/Java, and
then using the EDB server binaries, which should work fine.

> > Is this a change we need to make on the server end and then all the
> > installers will automatically install the file? It is present in all
> > Unix-like installs?
>
> AFAICS, it must at one time have been envisioned that sometimes
> extension code might like a nice dynloader; the whole way that
> backend/port/dynloader contains a version for every platform, and
> the configure script links the appropriate one into src/include with
> all the other .h files that would go into a -devel package, makes me
> _think_ that it'd be present in any install that used configure in
> the build. Anyone building a -devel package would have to go out
> of their way to exclude it but still package the other .h files.

Yes, it should. Looking at the 'install' Makefile rule in
include/Makefile I see:

cp $(srcdir)/*.h '$(DESTDIR)$(includedir_server)'/ || exit; \
chmod $(INSTALL_DATA_MODE) '$(DESTDIR)$(includedir_server)'/*.h || exit; \
for dir in $(SUBDIRS); do \
cp $(srcdir)/$$dir/*.h '$(DESTDIR)$(includedir_server)'/$$dir/ || exit; \
chmod $(INSTALL_DATA_MODE) '$(DESTDIR)$(includedir_server)'/$$dir/*.h || exit; \
done

This copies all the *.h files in src/include during install. If I look
at my Debian source install, I see the dynloader.h installed as
include/server/dynloader.h, which is what you want. In fact, there are
many include files installed in include/server:

c.h pg_config_ext.h pgtime.h postgres_ext.h
dynloader.h pg_config.h pg_trace.h postgres_fe.h
fmgr.h pg_config_manual.h plperl.h postgres.h
funcapi.h pg_config_os.h plpgsql.h ppport.h
getaddrinfo.h pg_getopt.h plpython.h rusagestub.h
getopt_long.h pgstat.h plpy_util.h windowapi.h
miscadmin.h pgtar.h port.h

> So I'd guess that Windows builds that don't use configure are probably
> the odd players out. But I don't have any contacts or name recognition
> to approach { EDB, OpenSCG, Postgres Pro } and find out how their
> internal build processes work, or whether they all end up lacking
> the file, or whether there is any single change that could be made
> in the PG source tree so that their builds would then all include it.

In fact, there is a single file that affects all the MSVC-based Windows
installers. All the MSVC build stuff happens in src/tools/msvc, and it
is mostly written in Perl. This is the corresponding line in the MSVC Perl
file Install.pm:

CopySetOfFiles(
'',
[ glob("src\\include\\*.h") ],
$target . '/include/server/');

So, for me, the big question is why dynloader.h isn't getting copied
into /include/server. (There is a mention the 'Makefile' about vpath
builds needing to copy dynloader.h manually --- is vpath being used for
the Windows installers somehow?)

Can you show me what files you have in /include/server, without your
copying the dynloader.h file manually? Where did you get that Windows
installer?

> > Also, I am very glad you are working on PL/Java. :-)
>
> Thanks! It has been interesting trying to get up to speed, both on
> how it technically works, and also on the development history, why it
> lives out-of-tree, and so on. (That question had puzzled me for a long
> time, and when I finally found the long 2006 -hackers thread,
> http://www.postgresql.org/message-id/44B3952B.2080401@tada.se
> I was like your genealogy-obsessed aunt finding a trunk in the attic. :)

Yes, it took us a long time to work out the logic of what should be
external.

> I can see that a lot of the considerations raised in that thread, both
> ways, are still pertinent today, plus with nine more years behind us
> to see how things have actually played out. Tom Lane was concerned about
> what would happen if Thomas's time for maintenance became scarce, and
> what seems to happen is someone like Johann Oskarsson, or yours truly,
> will step up, flounder a bit to get over the learning curve, and then
> carry the ball some distance. There is a bit of a barrier to entry,
> because PostgreSQL and Java are each large and deep and PL/Java has
> both, and for the future vigor of the project it seems important to
> find the ways to minimize that barrier. (I know I'm getting OT from
> dynloader here, but this other stuff has been on my mind a while.)
>
> That doesn't require reopening the in-tree question necessarily
> (I saw that Tom Lane was concerned about a buildfarm going all red
> because of a problem too few people could easily fix), but it would
> probably be very helpful to at least have some kind of _associated
> buildfarm_ so the main board might not go all red, but it would still
> be easy to see right away if a change was going to affect important
> out-of-tree components.
>
> That seems to have been a worthwhile idea nine years ago (I read that
> Andrew Dunstan _almost_ found the time to set it up then), and
> still today something like that would be very helpful. PL/Java still needs
> work on an easily-automatable suite of tests (that much, I'm working on),
> and once that's in place, the next obvious and valuable step would be to get
> it building continuously on the several major platforms, so it can turn red
> on some board that the team can easily see. I might ask for some help or
> suggestions on what are the currently most-favored ways to do that.

Yes, you can either add a buildfarm member of your own, or get someone
who is already on the buildfarm to build PL/Java as part of their build
process:

http://buildfarm.postgresql.org/

> I think that with more automated testing and CI, the barrier to entry on
> PL/Java contributions will be a lot lower, because it is much less
> intimidating to start poking at something when it is easy to see what
> happens.

Agreed.

--
Bruce Momjian <bruce(at)momjian(dot)us> http://momjian.us
EnterpriseDB http://enterprisedb.com

+ As you are, so once was I. As I am, so you will be. +
+ Roman grave inscription +