fork()-safety, thread-safety

From: Nico Williams <nico(at)cryptonector(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: fork()-safety, thread-safety
Date: 2017-10-05 22:02:22
Message-ID: 20171005220221.GZ1251@localhost
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

A thread on parallelization made me wonder so I took a look:

- src/bin/*/parallel.c uses threads on WIN32
- src/bin/*/parallel.c uses fork() on not-WIN32

(Ditto src/bin/pg_basebackup/pg_basebackup.c and
src/backend/postmaster/syslogger.c.)

A quick look at the functions called on the child side of fork()
makes me think that it's unlikely that the children here use
async-signal-safe functions only.

Why not use threads on all systems where threads are available when
we'd use threads on some such systems? If this code is thread-safe
on WIN32, why wouldn't it be thread-safe on POSIX? (Well, naturally
there may be calls to, e.g., getpwnam() and such that would not be
thread-safe on POSIX, and which might not exist on WIN32. But I
mean, aside from that, if synchronization is done correctly on WIN32,
what would stop that from being true on POSIX?)

- fork() is used in a number of places where execl() or execv() are
called immediately after (and exit() if the exec fails).

It would be better to use vfork() where available and _exit() instead
of exit().

Alternatively posix_spawn() should be used (which generally uses
vfork() or equivalent under the covers).

vfork() is widely demonized, but it's actually quite superior
(performance-wise) to fork() when all you want to do is exec-or-exit
since no page copying (COW or otherwise) needs be done when using
vfork().

It's actually safer to use vfork() because POSIX limits one to
async-signal-safe functions between fork() and exec-or-exit... With
fork(), where neither the parent nor the child immediately execs-or-
exits, it's too easy to fail to make sure that the code they execute
is fork-safe. Whereas with vfork() the fact that the parent (just
the one thread, incidentally, not all of them[*]) blocks until the
child execs-or-exits means it's impossible to fail to notice a
long-running child that does lots of fork-unsafe work.

It's safer still to use posix_spawn(), naturally.

In Unix-land it's standard practice to ignore the async-signal-safe
requirement when using fork() early on in a daemon's life to start
worker processes. This is fine, of course, though if we're using
CreateProcess*()/_spawn() on WIN32 anyways, it might be best to do the
equivalent on Unix and just spawn the children -- if nothing else, this
would reduce the likelihood of unintended divergence between WIN32 and
Unix.

Nico

[*] Actually, I do believe that on Solaris/Illumos vfork() stops all
threads in the parent, if I remember correctly anyways. Linux's and
NetBSD's vfork() only stops the one thread in the parent that called
it. I haven't checked other BSDs. There was a patch for NetBSD to
stop all threads in the parent, but I convinced the NetBSD community
to discard that patch.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Joshua D. Drake 2017-10-05 22:05:07 Re: search path security issue?
Previous Message David G. Johnston 2017-10-05 21:54:56 Re: search path security issue?