Re: [BUGS] 8.0.0beta1: -lpthread missing

OK, we now have thread compile failure reports on Debian and Slackware.

The config/acx_pthread.m4 script basically tests these:

acx_pthread_flags="pthreads none -Kthread -kthread lthread -pthread
-pthreads -mthreads pthread --thread-safe -mt pthread-config"

in that order and exits once it finds the first one that can
compile/link this:

AC_TRY_LINK([#include <pthread.h>],
[pthread_t th; pthread_join(th, 0);
pthread_attr_init(0); pthread_cleanup_push(0, 0);
pthread_create(0,0,0,0); pthread_cleanup_pop(0); ],
[acx_pthread_ok=yes])

Now, the big question is why -pthread can compile/link this successfully
but -pthread isn't enough to build a library that uses threads.

I just ran some tests on Debian 3.0 using Sourceforge's compile farm:

$ uname -a
Linux usf-cf-alpha-linux-1 2.2.20 #2 Wed Mar 20 19:57:28 EST 2002 alpha
unknown
$ cat x.c
main()
{
x2();
}
$ cat x2.c
#include <pthread.h>
x2()
{
pthread_t th;
pthread_key_t tkey;

pthread_join(th, 0);
pthread_attr_init(0);
pthread_cleanup_push(0, 0);
pthread_create(0,0,0,0);
pthread_cleanup_pop(0);
}
$ gcc -pthread -shared -Wl,-soname,x2 -o x2.so x2.c
$ gcc -o x x.c x2.so
x2.so: undefined reference to `pthread_create'
x2.so: undefined reference to `_pthread_cleanup_pop'
x2.so: undefined reference to `_pthread_cleanup_push'
x2.so: undefined reference to `pthread_join'
collect2: ld returned 1 exit status
$ # add -lpthread to lib
$ gcc -pthread -shared -Wl,-soname,x2 -o x2.so x2.c -lpthread
$ gcc -o x x.c x2.so
$ # add -pthread to binary
$ gcc -pthread -shared -Wl,-soname,x2 -o x2.so x2.c
$ gcc -pthread -o x x.c x2.so
$

This shows your issue exactly. Now, if I change x2.c to be main() and
use -pthread there:

$ cat x2.c
#include <pthread.h>
main()
{
pthread_t th;
pthread_key_t tkey;

pthread_join(th, 0);
pthread_attr_init(0);
pthread_cleanup_push(0, 0);
pthread_create(0,0,0,0);
pthread_cleanup_pop(0);
}
$ gcc -pthread -o x2 x2.c

it works, which is the test that is being done in config/acx_pthread.m4.

The basic problem is that the test script is testing for building
binaries with certain flags, not for building libraries that use
threads. This works fine for ecpg but not for libpq.

We can't just go and add -pthread to all the binary builds that use
libpq because we would then require user-built apps to also use that
flag too. We need to focus on building libpq properly so we don't need
anything special for building binaries using libpq.

So, this works:

$ gcc -shared -Wl,-soname,x2 -o x2.so x2.c -lpthread
$ gcc -o x x.c x2.so

while this does not:

$ gcc -pthread -shared -Wl,-soname,x2 -o x2.so x2.c
$ gcc -o x x.c x2.so
x2.so: undefined reference to `pthread_create'
x2.so: undefined reference to `_pthread_cleanup_pop'
x2.so: undefined reference to `_pthread_cleanup_push'
x2.so: undefined reference to `pthread_join'
collect2: ld returned 1 exit status

so if we just reordered the list of items in acx_pthread_flags and put
'pthread' first, your platform would work.

It is almost as though -pthread is ignored when making a shared lib; it
builds a shared object file that references threads fine without it:

$ gcc -shared -Wl,-soname,x2 -o x2.so x2.c
$ gcc -pthread -o x x.c x2.so

and that might be the core of the problem.

What I have done is to take a more agressive approach to the thread
flags than the script supported. Instead of exiting once the first
binary is built, the code will now use as many flags as posssible while
still building the binary. On your platform, that will mean -pthread
and -lpthread will be used.

Patch attached and applied.

---------------------------------------------------------------------------

Martin Mnstermann wrote:
> Bruce Momjian wrote:
> > OK, I have some more information. Basically, the config/acx_pthread.m4
> > manual page (http://www.gnu.org/software/ac-archive/htmldoc/acx_pthread.h=
> > tml)
> > says:
> > =09
> > NOTE: You are assumed to not only compile your program with these flags,
> > but also link it with them as well. e.g. you should link with
> > $PTHREAD_CC $CFLAGS $PTHREAD_CFLAGS $LDFLAGS ... $PTHREAD_LIBS $LIBS
> >
> > meaning if -lpthread is defined in PTHREAD_CFLAGS, it also has to be
> > included in the link. Now, from your reports you said -lpthread was in
> > PTHREAD_CFLAGS, so why isn't it also included in the libpq link line. I
> > need to see your link line output for libpq.
>
> OK, here are parts of the output from configure --enable-thread-safety
> on Debian:
> [...]
> checking for the pthreads library -lpthreads... no
> checking whether pthreads work without any flags... no
> checking whether pthreads work with -Kthread... no
> checking whether pthreads work with -kthread... no
> checking for the pthreads library -llthread... no
> checking whether pthreads work with -pthread... yes
> checking for joinable pthread attribute... PTHREAD_CREATE_JOINABLE
> checking if more special flags are required for pthreads... no
> checking for cc_r... gcc
> checking pthread.h usability... yes
> checking pthread.h presence... yes
> checking for pthread.h... yes
> [...]
> --> configure thays, thate the CFLAG -pthread is sufficient.
>
> Makefile.global:
> PTHREAD_CFLAGS = -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS
> PTHREAD_LIBS =
>
>
> Let's look what make is doing:
> [...]
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -I../../src/port -I../../src/include
> -D_GNU_SOURCE -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -c thread.c
> ar crs libpgport.a dirmod.o exec.o noblock.o path.o pipe.o pgsleep.o
> pgstrcasecmp.o sprompt.o thread.o
>
> [...]
>
> make[2]: Entering directory
> `/home/user/checkout/postgresql-8.0.0beta1/src/interfaces'
> make[3]: Entering directory
> `/home/user/checkout/postgresql-8.0.0beta1/src/interfaces/libpq'
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-auth.o fe-auth.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-connect.o
> fe-connect.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-exec.o fe-exec.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-misc.o fe-misc.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-print.o fe-print.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-lobj.o fe-lobj.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-protocol2.o
> fe-protocol2.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-protocol3.o
> fe-protocol3.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o pqexpbuffer.o
> pqexpbuffer.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o pqsignal.o pqsignal.c
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o fe-secure.o fe-secure.c
> rm -f dllist.c && ln -s ../../../src/backend/lib/dllist.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o dllist.o dllist.c
> rm -f md5.c && ln -s ../../../src/backend/libpq/md5.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o md5.o md5.c
> rm -f ip.c && ln -s ../../../src/backend/libpq/ip.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o ip.o ip.c
> rm -f wchar.c && ln -s ../../../src/backend/utils/mb/wchar.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o wchar.o wchar.c
> rm -f encnames.c && ln -s ../../../src/backend/utils/mb/encnames.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o encnames.o encnames.c
> rm -f noblock.c && ln -s ../../../src/port/noblock.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o noblock.o noblock.c
> rm -f pgstrcasecmp.c && ln -s ../../../src/port/pgstrcasecmp.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o pgstrcasecmp.o
> pgstrcasecmp.c
> rm -f thread.c && ln -s ../../../src/port/thread.c .
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -I. -I../../../src/include
> -D_GNU_SOURCE -I../../../src/port -DFRONTEND -c -o thread.o thread.c
> ar crs libpq.a `lorder fe-auth.o fe-connect.o fe-exec.o fe-misc.o
> fe-print.o fe-lobj.o fe-protocol2.o fe-protocol3.o pqexpbuffer.o
> pqsignal.o fe-secure.o dllist.o md5.o ip.o wchar.o encnames.o noblock.o
> pgstrcasecmp.o thread.o | tsort`
> ranlib libpq.a
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libpq.so.3
> fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o
> fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o
> dllist.o md5.o ip.o wchar.o encnames.o noblock.o pgstrcasecmp.o thread.o
> -L../../../src/port -lcrypt -lresolv -lnsl
> -Wl,-rpath,/opt/postgresql-800beta1/lib -o libpq.so.3.2
> rm -f libpq.so.3
> ln -s libpq.so.3.2 libpq.so.3
> rm -f libpq.so
> ln -s libpq.so.3.2 libpq.so
> make[3]: Leaving directory
> `/home/user/checkout/postgresql-8.0.0beta1/src/interfaces/libpq'
>
> So the shared library is built with "-pthread"
> [...]
>
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations initdb.o dirmod.o exec.o
> -L../../../src/interfaces/libpq -lpq -L../../../src/port
> -Wl,-rpath,/opt/postgresql-800beta1/lib -lpgport -lz -lreadline -lcrypt
> -lresolv -lnsl -ldl -lm -o initdb
>
> [--> results in linker error as described in the first posting]
>
>
> The problem can be resolved, if I modify my Makefile.global to
> PTHREAD_LIBS = -lpthread
>
> Then libpq.so is built with -lpthread (note the "l"):
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations -pthread -D_REENTRANT -D_THREAD_SAFE
> -D_POSIX_PTHREAD_SEMANTICS -fpic -shared -Wl,-soname,libpq.so.3
> fe-auth.o fe-connect.o fe-exec.o fe-misc.o fe-print.o fe-lobj.o
> fe-protocol2.o fe-protocol3.o pqexpbuffer.o pqsignal.o fe-secure.o
> dllist.o md5.o ip.o wchar.o encnames.o noblock.o pgstrcasecmp.o thread.o
> -L../../../src/port -lcrypt -lresolv -lnsl -lpthread
> -Wl,-rpath,/opt/postgresql-800beta1/lib -o libpq.so.3.2
>
>
> The link line of initdb is unchanged, with the difference that there is
> no error:
> gcc -O2 -fno-strict-aliasing -Wall -Wmissing-prototypes
> -Wmissing-declarations initdb.o dirmod.o exec.o
> -L../../../src/interfaces/libpq -lpq -L../../../src/port
> -Wl,-rpath,/opt/postgresql-800beta1/lib -lpgport -lz -lreadline -lcrypt
> -lresolv -lnsl -ldl -lm -o initdb
>
>
> So for me it looks like in my case it's not sufficient to have the
> compiler flag "-pthread", but also (or instead?) the linker flag
> "-lpthread".
> Maybe the case of a shared library using pthreads is not handled
> correctly in the m4 macro?
>
> Regards,
> Martin
>

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073