A Windows x64 port of MySQL

From: "Ken Camann" <kjcamann(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: A Windows x64 port of MySQL
Date: 2008-07-02 11:39:29
Message-ID: 63c05a820807020439s7729c2e0ra1515df9dfe8d9d1@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello everyone,

I don't really know much about contributing to projects, mailing
lists, or really anything that has to do with the UNIX culture (I am
not a UNIX fan). So forgive my ignorance of how this is supposed to
work.

Basically I am trying to compile postgres as a native x64 application.
I haven't gotten very far at all yet (I've fixed three very tiny
bugs) but I figured I'd try to get in touch with people on this list
before I do any more. Perhaps someone is already farther along than I
am, and if not, I can at least share my three tiny bug fixes with the
community and save you about 15-20 minutes :)

Basically the reason I started doing this was a misunderstanding; I
thought the ODBC driver (which is all I really need) always used libpq
to communicate with the server. As you may know, 32-bit and 64-bit
DLLs cannot load one another thus postgres being 32-bit forces
everything dependent on it to be 32-bit as well. While postgres works
fine in the SysWow64 environment on x64, there is no way to talk to it
from a 64-bit application because the ODBC driver is 32-bit. The
program I am writing is 64-bit only, thus I can't get database access.

Apparently from looking at the ODBC driver source code, it appears
that it is actually independent of postgres, does not explicitly on
depend on libpq, but uses sockets to directly communicate with the
server? Either way, I didn't know that so I started trying to build
an x64 version. I am aware that one of the contributers has created
an x64 driver here:
http://www.geocities.jp/inocchichichi/psqlodbc/index.html. It says
"really experimental" and I wonder what that means exactly, i.e., is
there work I should do to test it? There is nothing about it on the
main website (it was extremely hard to find that website).

Anywhere, here are the problems I found and the changes that a Windows
port maintainer might want to make:

To pg_config_os.h:
=============
On line 7:

change

#define _WIN32_WINNT 0x0500

to

#ifdef _WIN64
#define _WIN32_WINNT 0x0501
#else
#define _WIN32_WINNT 0x0500
#endif

Why: The first value (0x0500) is for Windows 2000, the second value
(0x0501) is for Windows XP or Server 2003. AFAIK 2000 did not support
x64 anyway so there is no harm in doing this (if _WIN64 is defined,
you can be assured the platform is at least 5.01). If you do not do
it, the IPPROTO_IPV6 macro in w2defs.h will not be defined. The fact
that this did not cause problems before probably means that the nature
of the #ifdef is different in w2defs.h for the 32-bit version of the
Win32 API. Microsoft's x64 port of Win32 requires it to be 0x0501 in
the winsock headers or IPPROTO_IPV6 is not defined. If that happens,
there will be an error in pqcomm.c

To s_lock.h:
=========
On line 809:

change

static __forceinline void
spin_delay(void)
{
/* see comment for gcc code. Same code, MASM syntax */
__asm rep npo;
}

to

#ifdef _WIN64
static __forceinline void
spin_delay(void)
{
__noop;
}
#else
static __forceinline void
spin_delay(void)
{
/* see comment for gcc code. Same code, MASM syntax */
__asm rep nop;
}
#endif

Why: None of Microsoft's x64 compilers (including 2008) support inline
assembly, so __asm is not recognized. This function implements a
delay for a spinlock by calling "rep; nop;" an instruction sequence
used by intel to delay and slow the processor speed down to the memory
bus speed. On AMD architectures it is the same as a regular nop. If
you can do "rep; nop;" this with the x64 compiler intrinsics (which
are in the compiler, unlike the ability the inline assembly), I
couldn't figure out how. I have an AMD processor anyway, so this is
the same as a regular nop for which there _is_ an intrinsic (__noop).
Note that as I type this, I realize that the #ifdef should not be
_WIN64 as this is not an inherent limitation of the architecture but
all current compilers. This should be conditioned on _MSVC_VER, the
compiler version number instead. I am not sure what the current
version is, however, so I'll just leave it like that.

To the postgres project:
=================
-Remove /DEF:".\Release\postgres\postgres.def" from the compiler
command line options.

Why: This is explained here: http://support.microsoft.com/kb/835326

Functions in the postgres project have a declspec(__dllexport)
declarator and the project uses a DEF file, but they shouldn't do
both. With the pre-x64 compilers, it is ok to define exports twice as
long as the definition is exactly the same. With the x64 compiler, it
is not ok (see link). Defining BUILDING_DLL will cause all exported
functions to have the declspec(__dllexport) declarator, so you can
either undefine BUILDING_DLL or remove the exporting option at the
project level. If you do not, first the linker will first produce
LNK4197 warnings, then lots of linker errors.

Other stuff:
========

The only remaining compile error issue has to do with the
_USE_32BIT_TIME_T workaround, which no longer works in x64 mode. I
haven't figured out how to deal with it, so I commented it out just to
see if I could get postgres.exe to compile. And it worked (it runs
too, but I didn't really do anything with it). So the postgres
project by itself works except for

-The time_t stuff
-There is one warning about a bit-wise shift that may behave
differently with 64-bit vs. 32-bit semantics...that would need to be
checked I just ignored it.
-There are endless numbers of the expected warning C4267 (conversion
from size_t to int/long: possible loss of data) because sizeof(size_t)
is 8 now and not sizeof(int) = 4. At first glance (I skimmed these)
its all from string processing code that assigns strlen(some_string)
to an int. There are hundreds of these, and probably all of them
don't matter. I assume it would only really matter if you did this to
a pointer, and perhaps that is happening somewhere (but I doubt it
since postgres runs fine on 64-bit POSIX OSes). These would be easy
to fix, but very annoying given the number of them.

So it won't be so hard after all to get it to run on Winx64...The big
issue is all the optional packages use POSIX libraries like GetText,
libxml, zlib...the binaries are all available but they're all 32-bit
DLLs so they can't be used...everything has to be rebuilt as a native
x64 DLL. That didn't matter to me since I'm not using any of them
(yet), and I was happy just to see how easily postgres built.

-Ken

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Zdenek Kotala 2008-07-02 12:01:21 Re: A Windows x64 port of PostgreSQL
Previous Message Mario Weilguni 2008-07-02 11:15:51 pg_compresslog/pg_decompresslog