Re: sidewinder has one failure

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Cc: Mikael Kjellström <mikael(dot)kjellstrom(at)mksoft(dot)nu>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: sidewinder has one failure
Date: 2020-01-03 16:56:37
Message-ID: 30892.1578070597@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I wrote:
> Amit Kapila <amit(dot)kapila16(at)gmail(dot)com> writes:
>> The problem we are seeing on this machine is that I think we have
>> seven files opened before we reach function set_max_safe_fds during
>> startup. Now, it is not clear to me why it is opening extra file(s)
>> during start-up as compare to other machines.

> Maybe it uses one of the semaphore implementations that consume a
> file descriptor per semaphore?

Hm, no, sidewinder reports that it's using SysV semaphores:

checking which semaphore API to use... System V

However, I tried building an installation that uses named POSIX
semaphores, by applying the attached hack on a macOS system.
And sure enough, this test crashes and burns:

2020-01-03 11:36:21.571 EST [91597] FATAL: insufficient file descriptors available to start server process
2020-01-03 11:36:21.571 EST [91597] DETAIL: System allows -8, we need at least 20.
2020-01-03 11:36:21.571 EST [91597] LOG: database system is shut down

Looking at "lsof" output for a postmaster with max_connections=10,
max_wal_senders=5 (the parameters set up by PostgresNode.pm), I see
that it's got 31 "PSXSEM" file descriptors, so the number shown here
is about what you'd expect. We might be able to constrain that down
a little further, but still, this test has no chance of working in
anything like its present form on a machine that needs file
descriptors for semaphores. That's a supported configuration, even
if not a recommended one, so I don't think it's okay for the test
to fall over.

(Hmm ... apparently, we have no buildfarm members that use such
semaphores and are running the TAP tests, else we'd have additional
complaints. Perhaps that's a bad omission.)

Anyway, it remains unclear exactly why sidewinder is failing, but
I'm guessing it has a few more open files than you expected. My
macOS build has a few more than I can account for in my caffeine-
deprived state, too. One of them might be for bonjour ... not sure
about some of the rest. Bottom line here is that it's hard to
predict with any accuracy how many pre-opened files there will be.

regards, tom lane

diff --git a/src/template/darwin b/src/template/darwin
index f4d4e9d7cf..98331be22d 100644
--- a/src/template/darwin
+++ b/src/template/darwin
@@ -23,11 +23,4 @@ CFLAGS_SL=""
# support System V semaphores; before that we have to use named POSIX
# semaphores, which are less good for our purposes because they eat a
# file descriptor per backend per max_connection slot.
-case $host_os in
- darwin[015].*)
USE_NAMED_POSIX_SEMAPHORES=1
- ;;
- *)
- USE_SYSV_SEMAPHORES=1
- ;;
-esac

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Stephen Frost 2020-01-03 17:01:23 Re: backup manifests
Previous Message Robert Haas 2020-01-03 16:51:06 Re: backup manifests