RE:PostgreSQL 11 beta1 on AIX 7.2 : 2 failures in 32bit mode

Hi Tom,

We are using Python 2.7.12 , built on AIX 6.1 with GCC .

Looking at its build log, I see about the traces of the configure:
64bit: checking whether to enable large file support... no
32bit: checking whether to enable large file support... yes

/home2/freeware/include/python2.7/pyconfig-ppc32.h :
/* This must be defined on AIX systems to enable large file support. */
#define _LARGE_FILES 1

# cat /opt/freeware/include/python2.7/pyconfig.h
...
#if defined(__64BIT__)
#include "pyconfig-ppc64.h"
#else
#include "pyconfig-ppc32.h"
#endif
...

Looking at tests, I see:
32bit:
[194/396/5] test_largefile
test_largefile skipped -- filesystem does not have largefile support

343 tests OK.
12 tests failed:
test___all__ test_capi test_ctypes test_hotshot test_httpservers
test_locale test_resource test_ssl test_tcl test_ttk_textonly
test_urllib2 test_urllib2_localnet
41 tests skipped:
4 skips unexpected on aix6:
test_bsddb185 test_gdb test_largefile test_spwd

However, this depends on ulimit.
On AIX, several resources are not large or unlimited like Linus does.

With standard ulimit:
# ulimit -a
file size (blocks, -f) 1048575
# ./python ./Lib/test/test_largefile.py
unittest.case.SkipTest: filesystem does not have largefile support

With unlimited:
# ulimit -a
file size (blocks, -f) unlimited
# ./python ./Lib/test/test_largefile.py
Ran 18 tests. OK (skipped=1)
test_seekable (__main__.BuiltinLargeFileTest) ... skipped "builtin file doesn't have seekable()"

So. I see no problem with LARGE FILES with Python on AIX.

Cordialement,

Tony Reix

ATOS / Bull SAS
ATOS Expert
IBM Coop Architect & Technical Leader
Office : +33 (0) 4 76 29 72 67
1 rue de Provence - 38432 Échirolles - France
www.atos.net

________________________________________
De : Tom Lane [tgl(at)sss(dot)pgh(dot)pa(dot)us]
Envoyé : vendredi 1 juin 2018 01:23
À : Noah Misch
Cc : REIX, Tony; Alvaro Herrera; PostgreSQL-development; APEKE, SENA (ext); Peter Eisentraut; Anthony Bykov
Objet : Re: PostgreSQL 11 beta1 on AIX 7.2 : 2 failures in 32bit mode

Noah Misch <noah(at)leadboat(dot)com> writes:
> On Thu, May 31, 2018 at 10:28:12AM -0400, Tom Lane wrote:
>> I wondered why the existing 32-bit AIX buildfarm machines aren't showing
>> problems, but looking closer at them, they are manually forcing
>> _LARGE_FILES, which probably is masking things:
>> 'config_env' => {
>> 'CC' => 'wrap-gcc -D_THREAD_SAFE=1 -D_LARGE_FILES=1 -maix32',
>> Noah, why'd you do that, and would you be willing to remove it? IMO
>> Postgres should work without that.

> I did that to work around a problem like the one articulated upthread.
> Specifically, a 64-bit build w/ plpython failed:
> ...
> Today's "configure" test concludes that we don't need _LARGE_FILES, because
> off_t is 64-bit ("long", specifically) in this configuration. The trouble
> arises when Python.h does cause _LARGE_FILES to be defined.

Ugh. That's a pretty crummy decision on their part, although maybe there
was no better alternative.

This does not seem like it explains Tony's problem with AIX 32-bit,
though, as you'd think all concerned would agree _LARGE_FILES needs
to be 1 in that case.

> At the time I added the workaround, I scratched down these candidates for a
> proper fix:

> 1. Add "configure" test to determine whether Python defines _LARGE_FILES.
> When it does, define it ourselves at the top of each Python-associated
> source file.

That would make aspects of our extension ABI dependent on whether one had
configured with --enable-python, which would be surprising to say the
least.

> 2. Define _LARGE_FILES unconditionally on AIX. That is, adopt the workaround
> as permanent.

Perhaps. I wonder though whether this is really an AIX-only problem.
(In particular, I wonder whether Python.h is clobbering _FILE_OFFSET_BITS
on other platforms.) There's a comment in Autoconf's AC_SYS_LARGEFILE
that suggests it is, but ...

> 3. Define _LARGE_FILES unconditionally. This should be harmless, but I
> wouldn't tend to back-patch it.

It seems like variants of this issue should exist in all branches,
so I'm not really happy with taking a fix we're scared to back-patch.
If we were willing to do so, though, this might be OK. Seems like there
are three possibilities:
* Defining _LARGE_FILES does something good, in which case we want it.
* Defining _LARGE_FILES does nothing.
* Defining _LARGE_FILES does something bad ... but it's hard to see
how that could be.

> 4. Include system headers that react to _LARGE_FILES before including
> Python.h. This is fragile; the list of affected headers may change.

Yeah, that seems fairly unworkable, though I wonder whether the
include-ordering advice in plpython.h isn't basically meant to achieve
this result.

We're still left with the question of why Tony is having a problem.
I wonder whether his build of Python.h is doing something strange
with _LARGE_FILES.

regards, tom lane