Re: [PATCH] Make ENOSPC not fatal in semaphore creation

From: Mikhail <mp39590(at)gmail(dot)com>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [PATCH] Make ENOSPC not fatal in semaphore creation
Date: 2021-10-23 15:40:44
Message-ID: YXQs/DSCM+FUiGrF@edge.lab.local
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

On Fri, Oct 22, 2021 at 09:00:31PM -0400, Tom Lane wrote:
> I tried this on an OpenBSD 6.0 image I had handy. The good news is
> that it works, and I can successfully start the postmaster with a lot
> of semaphores (I tried with max_connections=10000) without any special
> system configuration. The bad news is it's *slow*. It takes the
> postmaster over a minute to start up at 10000 max_connections, and
> also about 15 seconds to shut down. The regression tests also appear
> noticeably slower, even at the default max_connections=100. I'm
> afraid that those "lots of tiny mappings" that Thomas noted have
> a nasty impact on our process launch times, since the kernel
> presumably has to do work to clone them into the child process.
> Now this lashup that I'm testing on is by no means well suited for
> performance tests, so maybe my numbers are bogus. Also, maybe it's
> better in more recent OpenBSD releases. But I think we need to take a
> harder look at performance before we decide that it's okay to change
> the default semaphore type for this platform.

I got following results for "time make installcheck" on a laptop with
OpenBSD 7.0 (amd64):

POSIX (max_connections=100) (default): 1m32.39s real 0m03.82s user 0m05.75s system
POSIX (max_connections=10000): 2m13.11s real 0m03.56s user 0m07.06s system

SysV (max_connections=100) (default): 1m24.39s real 0m03.30s user 0m04.94s system
SysV (max_connections=10000): failed to start
after sysctl tunning:
SysV (max_connections=10000): 1m47.51s real 0m03.78s user 0m05.61s system

I can confirm that start and stop of the server was slower in POSIX
case, but not terribly different (seconds, not a minute, as in your

As the OpenBSD developers said - those who use OpenBSD are never after a
good performance, the system has a lot of bottlenecks except IPCs.

I see following reasons to switch from SysV to POSIX:

- consistency in the ports tree, all major ports use POSIX, it means
better testing of the API
- as already pointed out - OpenBSD isn't about performance, and the
results for default max_connections are pretty close
- crash recovery with the OS defaults is automatic and don't require DBA
intervention and knowledge of ipcs and ipcrm
- higher density is available without system tuning

The disadvantage is in a worse performance for extreme cases, but I'm
not sure OpenBSD is used for them in production.

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2021-10-23 15:49:44 Re: XTS cipher mode for cluster file encryption
Previous Message Bruce Momjian 2021-10-23 15:29:05 Re: XTS cipher mode for cluster file encryption