Remove pthread_is_threaded_np() checks in postmaster

From: "Tristan Partin" <tristan(at)neon(dot)tech>
To: "pgsql-hackers" <pgsql-hackers(at)postgresql(dot)org>
Cc: "Alexander Bayandin" <alexander(at)neon(dot)tech>
Subject: Remove pthread_is_threaded_np() checks in postmaster
Date: 2024-01-23 19:20:15
Message-ID: CYMBV0OT7216.JNRUO6R6GH86@neon.tech
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

These checks are not effective for what they are trying to prevent.
A recent commit[0] in libcurl when used on macOS has been tripping the
pthread_is_threaded_np() check in postmaster.c for
shared_preload_libraries entries which use libcurl (like Neon). Under
the hood, libcurl calls SCDynamicStoreCopyProxies[1], which apparently
causes the check to fail.

Attached is a patch to remove the check, and a minimal reproducer
(curlexe.zip) for others to run on macOS.

Here are some logs:

Postgres working as expected:
> $ LC_ALL="C" /opt/homebrew/opt/postgresql(at)16/bin/postgres -D /opt/homebrew/var/postgresql(at)16
> 2024-01-22 23:18:51.203 GMT [50388] LOG: starting PostgreSQL 16.1 (Homebrew) on aarch64-apple-darwin23.2.0, compiled by Apple clang version 15.0.0 (clang-1500.1.0.2.5), 64-bit
> 2024-01-22 23:18:51.204 GMT [50388] LOG: listening on IPv6 address "::1", port 5432
> 2024-01-22 23:18:51.204 GMT [50388] LOG: listening on IPv4 address "127.0.0.1", port 5432
> 2024-01-22 23:18:51.205 GMT [50388] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
> 2024-01-22 23:18:51.207 GMT [50391] LOG: database system was shut down at 2023-12-21 23:12:10 GMT
> 2024-01-22 23:18:51.211 GMT [50388] LOG: database system is ready to accept connections
> ^C2024-01-22 23:18:53.797 GMT [50388] LOG: received fast shutdown request
> 2024-01-22 23:18:53.798 GMT [50388] LOG: aborting any active transactions
> 2024-01-22 23:18:53.800 GMT [50388] LOG: background worker "logical replication launcher" (PID 50394) exited with exit code 1
> 2024-01-22 23:18:53.801 GMT [50389] LOG: shutting down
> 2024-01-22 23:18:53.801 GMT [50389] LOG: checkpoint starting: shutdown immediate
> 2024-01-22 23:18:53.805 GMT [50389] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.002 s, sync=0.001 s, total=0.005 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB; lsn=0/4BE77E0, redo lsn=0/4BE77E0
> 2024-01-22 23:18:53.809 GMT [50388] LOG: database system is shut down

Postgres not working with attached extension preloaded:
> $ echo shared_preload_libraries=curlexe >> /opt/homebrew/var/postgresql(at)16/postgresql.conf
> $ LC_ALL="C" /opt/homebrew/opt/postgresql(at)16/bin/postgres -D /opt/homebrew/var/postgresql(at)16
> 2024-01-22 23:19:01.108 GMT [50395] LOG: starting PostgreSQL 16.1 (Homebrew) on aarch64-apple-darwin23.2.0, compiled by Apple clang version 15.0.0 (clang-1500.1.0.2.5), 64-bit
> 2024-01-22 23:19:01.110 GMT [50395] LOG: listening on IPv6 address "::1", port 5432
> 2024-01-22 23:19:01.110 GMT [50395] LOG: listening on IPv4 address "127.0.0.1", port 5432
> 2024-01-22 23:19:01.111 GMT [50395] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
> 2024-01-22 23:19:01.113 GMT [50395] FATAL: postmaster became multithreaded during startup
> 2024-01-22 23:19:01.113 GMT [50395] HINT: Set the LC_ALL environment variable to a valid locale.
> 2024-01-22 23:19:01.114 GMT [50395] LOG: database system is shut down

Same as previous, but without IPv6 support in libcurl:
> $ DYLD_LIBRARY_PATH=/opt/homebrew/opt/curl-without-ipv6/lib LC_ALL="C" /opt/homebrew/opt/postgresql(at)16/bin/postgres -D /opt/homebrew/var/postgresql(at)16
> 2024-01-22 23:23:17.151 GMT [50546] LOG: starting PostgreSQL 16.1 (Homebrew) on aarch64-apple-darwin23.2.0, compiled by Apple clang version 15.0.0 (clang-1500.1.0.2.5), 64-bit
> 2024-01-22 23:23:17.152 GMT [50546] LOG: listening on IPv6 address "::1", port 5432
> 2024-01-22 23:23:17.152 GMT [50546] LOG: listening on IPv4 address "127.0.0.1", port 5432
> 2024-01-22 23:23:17.152 GMT [50546] LOG: listening on Unix socket "/tmp/.s.PGSQL.5432"
> 2024-01-22 23:23:17.155 GMT [50549] LOG: database system was shut down at 2024-01-22 23:23:10 GMT
> 2024-01-22 23:23:17.158 GMT [50546] LOG: database system is ready to accept connections
> ^C2024-01-22 23:23:26.997 GMT [50546] LOG: received fast shutdown request
> 2024-01-22 23:23:26.998 GMT [50546] LOG: aborting any active transactions
> 2024-01-22 23:23:27.000 GMT [50546] LOG: background worker "logical replication launcher" (PID 50552) exited with exit code 1
> 2024-01-22 23:23:27.000 GMT [50547] LOG: shutting down
> 2024-01-22 23:23:27.001 GMT [50547] LOG: checkpoint starting: shutdown immediate
> 2024-01-22 23:23:27.002 GMT [50547] LOG: checkpoint complete: wrote 3 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.001 s, sync=0.001 s, total=0.003 s; sync files=2, longest=0.001 s, average=0.001 s; distance=0 kB, estimate=0 kB; lsn=0/4BE78D0, redo lsn=0/4BE78D0
> 2024-01-22 23:23:27.005 GMT [50546] LOG: database system is shut down

[0]: https://github.com/curl/curl/commit/8b7cbe9decc205b08ec8258eb184c89a33e3084b
[1]: https://developer.apple.com/documentation/systemconfiguration/1517088-scdynamicstorecopyproxies

--
Tristan Partin
Neon (https://neon.tech)

Attachment Content-Type Size
curlexe.zip application/zip 839 bytes
v1-0001-Remove-pthread_is_threaded_np-checks-in-postmaste.patch text/x-patch 4.3 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2024-01-23 19:38:54 Re: pgsql: Add better handling of redundant IS [NOT] NULL quals
Previous Message Andy Fan 2024-01-23 19:18:18 Re: Shared detoast Datum proposal