Mingw task for Cirrus CI

From: Melih Mutlu <m(dot)melihmutlu(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Mingw task for Cirrus CI
Date: 2022-02-25 16:44:27
Message-ID: CAGPVpCSKS9E0An4=e7ZDnme+y=WOcQFJYJegKO8kE9=gh8NJKQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi All,

I've been working on adding Windows+MinGW environment into cirrus-ci tasks
(discussion about ci is here [1]).
It uses MSYS2 to set the environment. UCRT is chosen as C standard library,
instead of MSVCRT.
The task configures postgres with features that are available in MSYS2 (see
available packages [2]) and tap tests are enabled.
I already added the necessary docker image, you can find the related PR at
[3] and a successful cirruc-ci run with these changes at [4].
Attached patch adds a task runs on Windows with MinGW for cirrus-ci

However, I cannot run configure with --with-python, --with-perl or
--with-tcl.
There are two issues I encountered while trying to enable. e.g. for
--with-python

1- python_ldlibrary is set to "libpython3.9.dll.a". So the related line in
configure cannot retrieve "ldlibrary"
2- src/pl/plpython/Makefile looks under "C:/Windows/system32/" for
PYTHONDLL. gendef cannot open that file, give an error like " failed to
open()" when creating a .def file.

To overcome those, I added the correct pattern to extract ldlibrary by
appending "-e 's/\.dll.a$//'" at the end of the related line.
Then, I tried to use python dll from MSYS instead of windows/system32 by
changing PYTHONDLL path to "/ucrt64/bin/libpython3.9.dll". I'm not sure if
this is the correct dll.
Here is the diff of these two changes:

diff --git a/configure b/configure
index f3cb5c2b51..42ea580442 100755
--- a/configure
+++ b/configure
@@ -10536,7 +10536,7 @@ python_libdir=`${PYTHON} -c "import sysconfig;
print(' '.join(filter(None,syscon
python_ldlibrary=`${PYTHON} -c "import sysconfig; print('
'.join(filter(None,sysconfig.get_config_vars('LDLIBRARY'))))"`

# If LDLIBRARY exists and has a shlib extension, use it verbatim.
-ldlibrary=`echo "${python_ldlibrary}" | sed -e 's/\.so$//' -e 's/\.dll$//'
-e 's/\.dylib$//' -e 's/\.sl$//'`
+ldlibrary=`echo "${python_ldlibrary}" | sed -e 's/\.so$//' -e 's/\.dll$//'
-e 's/\.dylib$//' -e 's/\.sl$//' -e 's/\.dll.a$//'`
if test -e "${python_libdir}/${python_ldlibrary}" -a
x"${python_ldlibrary}" != x"${ldlibrary}"
then
ldlibrary=`echo "${ldlibrary}" | sed "s/^lib//"`
diff --git a/src/pl/plpython/Makefile b/src/pl/plpython/Makefile
index a83ae8865c..4254ef94d7 100644
--- a/src/pl/plpython/Makefile
+++ b/src/pl/plpython/Makefile
@@ -61,7 +61,8 @@ INCS = plpython.h \
ifeq ($(PORTNAME), win32)

pytverstr=$(subst .,,${python_version})
-PYTHONDLL=$(subst \,/,$(WINDIR))/system32/python${pytverstr}.dll
+#PYTHONDLL=$(subst \,/,$(WINDIR))/system32/python${pytverstr}.dll
+PYTHONDLL=/ucrt64/bin/libpython3.9.dll

OBJS += libpython${pytverstr}.a

In the end, make check-world still fails, even though I was able to run
configure and make without any obvious error.
I see bunch of errors in tests like:
+ERROR: language "plpython3u" does not exist
+HINT: Use CREATE EXTENSION to load the language into the database.

Here is the logs from failed ci run:
https://api.cirrus-ci.com/v1/artifact/task/4645682031099904/log/build/src/pl/plpython/regression.diffs

Any thoughts on how postgres can be built with --with-python etc. on mingw?

Best,
Melih

[1]
https://www.postgresql.org/message-id/flat/20211001222752.wrz7erzh4cajvgp6%40alap3.anarazel.de
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.postgresql.org%2Fmessage-id%2Fflat%2F20211001222752.wrz7erzh4cajvgp6%2540alap3.anarazel.de&data=04%7C01%7CMelih.Mutlu%40microsoft.com%7C7f4780e35ab34e2f38d808d9f7d008e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637813295184245763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=HkjuBWcQzmiyIyoCuIfKCzAEF9ME6SpGOYjBJMFhQvw%3D&reserved=0>
[2] https://packages.msys2.org/package/
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpackages.msys2.org%2Fpackage%2F&data=04%7C01%7CMelih.Mutlu%40microsoft.com%7C7f4780e35ab34e2f38d808d9f7d008e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637813295184245763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=VqfdnrxHv15FUGFXgnNaMli5n2R1tnPPglGq4xWqND0%3D&reserved=0>
[3] https://github.com/anarazel/pg-vm-images/pull/8
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fanarazel%2Fpg-vm-images%2Fpull%2F8&data=04%7C01%7CMelih.Mutlu%40microsoft.com%7C7f4780e35ab34e2f38d808d9f7d008e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637813295184245763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=IASIgPhLd0%2Fvh1VSn9EC%2BCsH42riE%2Bt6gQPObOb0QmI%3D&reserved=0>
[4] https://cirrus-ci.com/build/4999469182746624
<https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcirrus-ci.com%2Fbuild%2F5013577319841792&data=04%7C01%7CMelih.Mutlu%40microsoft.com%7C7f4780e35ab34e2f38d808d9f7d008e0%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637813295184245763%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=hMGCtgm1uVBWzFHXPEl%2FiaavzWbEqxAuhmKjFcjH%2BF0%3D&reserved=0>

Attachment Content-Type Size
0001-Added-Windows-with-MinGW-environment-in-Cirrus-CI.patch application/octet-stream 4.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andres Freund 2022-02-25 16:47:22 Re: Add parameter jit_warn_above_fraction
Previous Message Julien Rouhaud 2022-02-25 16:43:15 Re: Expose JIT counters/timing in pg_stat_statements