pg_threads.h take II

Hi,

Here's another go at port/pg_threads.h, a C11 <threads.h>-like
interface, with some patches to use it (some just sketch-quality).
Previous work:

https://www.postgresql.org/message-id/flat/CA%2BhUKGLtmexrpMtxBRLCVePqV_dtWG-ZsEbyPrYc%2BNBB2TkNsw%40mail.gmail.com

We still can't require C11 <threads.h> yet, though the interface has a
clear future upgrade/remap path to do that eventually. Since I last
wrote about this topic, we gained the ability to count on C11
thread_local being available (it's a language/compiler feature and
doesn't need a <threads.h> header or library), and partly because of
that, but also because I don't think anyone really likes it, I ripped
out that tss_t stuff. I came up with a narrower API to get
thread-exit cleanup callbacks if you need them for your thread_local
resources, same pthread_key_t/FlsAlloc stuff underneath it but without
having to get lost in the weeds of <threads.h> conformance.

v1-0001-ecpg-Fix-auto_mem-cleanup-on-thread-exit.patch

Already discussed in another thread, a couple of existing ECPG
thread-exit cleanup bugs need to be fixed (bugs that cancel each other
out in master, but leak, and with my patches the leak was fixed in
passing but then a double-free would crash):

https://www.postgresql.org/message-id/flat/CA%2BhUKGLv202ndx%3DkP1fZJapNcbfnpq-rezHNyt%2BcQ4pdxr1akg%40mail.gmail.com

v1-0002-port-Provide-minimal-pg_threads.h-API.patch

The main thing I'm wondering about is how to make the error reporting
a bit nicer. Neither pthreads nor C11 threads set errno, but it's
nice if you can use %m. Hmm.

v1-0003-port-Use-pg_threads.h-API-for-pg_localconv_r.patch

Trivial.

v1-0004-ecpg-Improve-variable-name.patch
v1-0005-ecpg-Use-pg_threads.h.patch

These demonstrate the replacement of pthread_key_t with thread_local,
and where needed, pg_thrd_atexit_t. That's the narrower thing I came
up with to replace pthread_t/tss_t with something that more directly
models what we want, I think.

I could do pg_tss_t, but it takes a lot of extra book-keeping code to
make Windows FlsAlloc match the spec.

v1-0006-pgbench-Use-pg_threads.h.patch

Kills another local thread-porting abstraction.

v1-0007-libpq-Use-pg_threads.h.patch

Kills another local thread-porting abstraction.

v1-0008-pg_basebackup-Use-pg_threads.h.patch

Kills the fork() mode used for Unix, harmonises Unix and Windows.

v1-0009-pg_dump-Remove-TerminateThread-call.patch

This one already has a discussion thread (I figured it might be
another back-patchable issue, IDK), and it looks like the solution I
used here isn't what we want to go with for the future, but to do it
the way Heikki and Jelte are suggesting will probably require deciding
how to access atomics or at least compiler barrires in front-end code,
which I'll look into this week.

Consider this approach a stand-in for now, as I needed to get rid of a
TerminateThread() call somehow. We can't have that in pg_threads.h
for various reasons (it's all kinds of undefined).

https://www.postgresql.org/message-id/flat/CA%2BhUKGJgO%3Do-vLFahGdR2WesuX3h1-0j%3Da8z72fChc-MG1Hveg%40mail.gmail.com

v1-0010-pg_dump-Block-signals-during-signal-handler.patch

Not very interesting, I was just triggered by steampunk Unix
anachronisms in passing... an experimental patch further down
replaces all of this anyway.

v1-0011-pg_dump-Use-pg_threads.h-and-remove-fork-path.patch

Kills the fork() mode used for Unix, harmonises Unix and Windows.
After this patch, the cancellation stuff admittedly looks pretty
shaky, but I wanted to separate the fork()-ectomy from any larger
refactoring, so bear with me...

Next I was planning to rip out the pipes that fork()-mode needed and
use an in-memory work queue, and probably make a reusable pg_thrd_pool
instead of open coding it here, but it turned out that Bryan was
already way ahead of me there:

https://www.postgresql.org/message-id/flat/8c712d76-ecf7-4749-a6d8-dddc01f298ec(at)gmail(dot)com

Luckily it looks like he started working at that end of the problem
and I started with this fork()-ectomy/harmonisation piece, so I
think/hope that we can fit those efforts together quite easily... I
will look at his patches this week.

v1-0012-pg_signal_processor-Infrastructure-for-cleanup.patch
v1-0013-fe_utils-Provide-cancel_set-for-fast-quit-paths.patch
v1-0014-pg_dump-Refactor-connection-cancel-management.patch

I considered lots of different ways to make pg_dump cleanup better,
and this is the approach that *seemed* to be roughly along the lines
that Jelte suggested he wanted for various reasons, over in the
TerminateThread() thread. Namely, make it so that Unix and Windows
both run your cleanup code on a thread, but also force Windows to
serialise the handlers, so that the semantics match and we can escape
from the constraints of async-signal-safe signal handlers. On Unix it
has a thread sitting there waiting for you to hit ^C, just looping
over sigwait() and calling handlers serially, that are then free to
use arbitrarily complex code before _exit() turns out the lights.

Warning/disclaimer: 0012-0014 are only experimental sketches trying
stuff out, and might not work that well in various ways, I just wanted
to get your take on this general direction. Could skin this in lots
of other ways too and since you guys are actively hacking on this area
you might already have another idea in mind or even patches?

After these patches, there are a few places left that use raw Windows
threading APIs, but they're places that are creating threads to solve
signaling and I/O problems that have better solutions, topics for
another email (but briefly I mean: the interrupts/latches refactoring
patch can kill "our" signal-emulation threads, and support for
overlapped I/O can allow multiplexing pipe I/O with latches without
intermediate data-pumping threads (cf Run::IPC which also has that
kind of Windows-hates-select() problem), and support for process-exit
WaitEvents can kill another helper thread, ... yada yada).