meson vs mingw, plpython, readline and other fun

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org, Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net>
Subject: meson vs mingw, plpython, readline and other fun
Date: 2022-09-28 02:27:24
Message-ID: 20220928022724.erzuk5v4ai4b53do@awork3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Melih mentioned on IM that while he could build postgres with meson on windows
w/ mingw, the tests didn't run.

Issues:

- The bit computing PATH to the temporary installation for running tests only
dealt with backward slashes in paths on windows, because that's what
python.org python uses by default. But a msys ucrt python returns forward
slashes. Trivial fix.

I didn't encounter this because I'd used a meson from git, which thus didn't
have msys's patch to return a different prefix.

This made pg_regress/isolationtester tests other than the main regression
tests pass.

- I'd only passed in a fake HOST_TUPLE when building pg_regress, oops.

I don't think it makes sense to come up with a config.guess compatible name
- they're quite random. And we can't rely on shell to work when targetting
msvc. The attached patch does:

# Need make up something roughly like x86_64-pc-mingw64. resultmap matches on
# patterns like ".*-.*-mingw.*". We probably can do better, but for now just
# replace 'gcc' with 'mingw' on windows.
host_tuple_cc = cc.get_id()
if host_system == 'windows' and host_tuple_cc == 'gcc'
host_tuple_cc = 'mingw'
endif
host_tuple = '@0(at)-@1(at)-@2@'.format(host_cpu, host_system, host_tuple_cc)

which I don't perfectly like (e.g. clang also kind of works on windows), but
it seems ok enough for now. I suspect we'd need a bunch of other changes to
make clang on windows work.

This made the main pg_regress tests pass.

- I had not added the logic to not use existing getopt on mingw, causing tap
tests to fail. Fixing that didn't immediately work because of duplicate
symbols - because I hadn't copied over -Wl,--allow-multiple-definition.

"Contrary" to the comment in src/template/win32 it doesn't appear to be
needed for libpq and pgport - likely because for the meson build an export
file is generated (needed for msvc anyway, I didn't think to disable it for
mingw). But since we need to be able to override getopt(), we obviously
need --allow-multiple-definition anyway.

- This lead me to try to also add -Wl,--disable-auto-import. However, that
caused two problems.

1) plpython tests started to fail, due to not finding Pg_magic_func in
plpython3.dll. This confounded me for quite a while. It worked for every
other extension .dll? A lot of looking around lead me to define
#define PGDLLEXPORT __declspec (dllexport)
for mingw as well. For mingw we otherwise end up with
#define PGDLLEXPORT __attribute__((visibility("default")))
which works.

As far as I can tell __attribute__((visibility("default"))) works as long as
as there's no declspec(dllexport) symbol in the same dll. If there is, it
stops working. Ugh.

I don't see a reason not to define PGDLLEXPORT as __declspec(dllexport)
for mingw as well?

I suspect this is an issue for autoconf mingw build as well, but that
fails to even configure - there's no coverage on the BF for it I think.

This made plpython's test pass (again).

2) psql failed to build due to readline. I hadn't implemented disabling it
automatically. Somewhat luckily - turns out it actually works (as long as
--disable-auto-import isn't used), including autocomplete!

The issue afaict is that while readline has an import library, functions
aren't "annotated" with __declspec(dllimport), thus without
--enable-auto-import the references are assumed to be local, and thus
linking fails.

It's possible we could "fix" this by declaring the relevant symbols
ourselves or such. But for now just adding --enable-auto-import to the
flags used to link to readline seems saner?

I think it'd be very cool to finally have a working readline on windows.

Unfortunately IO::Pty isn't installable on windows, it'd have been
interesting to see how well that readline works.

- Before I updated mingw, interactive psql didn't show a prompt, making me
think something was broken. That turned out to be the same in an autoconf
build. When inside the msys terminal (mintty) isatty returned 0, because of
some detail of how it emulates ttys. After updating mingw that problem is
gone.

I included this partially so I have something to find in email next time I
search for mintty and isatty :)

With these things fixed, postgres built and ran tests successfully! With
nearly all "relevant" dependencies:
icu, libxml, libslt, lz4, nls, plperl, plpython, pltcl, readline, ssl, zlib,
zstd

Missing are gss and uuid. Both apparently work on windows, but they're not in
the msys repository, and I don't feel like compiling them myself.

Only 5 tests skipped:
- recovery/017_shm - probably not applicable
- recovery/022_crash_temp_files - I don't think it's ok that this test skips,
but that's for another thread
- psql/010_tab_completion - IO::Pty can't be installed
- psql/010_cancel - IO::Pty can't be installed
- ldap/001_auth - doesn't know how to find slapd on windows

Stared too long at the screen to figure all of this out. Food next. I'll clean
the patches up later tonight or tomorrow morning.

Greetings,

Andres Freund

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2022-09-28 02:46:38 Re: Assign TupleTableSlot->tts_tableOid duplicated in tale AM.
Previous Message Peter Smith 2022-09-28 02:04:00 Re: GUC tables - use designated initializers