BUG #14588: Postgres process blocked on semop

The following bug has been logged on the website:

Bug reference: 14588
Logged by: Chenhong Liu
Email address: diabloneo(at)gmail(dot)com
PostgreSQL version: 9.2.16
Operating system: CentOS 7.2 1511
Description:

We run a distributed application which storing data in pgsql. This
application is designed to do some route works during 00:00 to 6:00 every
day. I found that, postgres processes are all blocked on semop function,
which means they are waiting for sysv semaphores. I'm not sure if this is a
bug or we misconfigured pgsql and OS.

Techinque information:

30 hosts running the application.
3 hosts running the postgresql servers, one master and two host-stanby
servers.

pgsql config:
max_connections = 1000
shared_buffers = 512MB

OS kernel.sem
250 32000 32 128

During the daytime, there are about 300 postgres processes running, work as
expected. On 00:00 each day, there will be about 200 more postgresql
processes created, now totally about 500. And from that time, thoese working
processes are all blocked on semop function. I check this using pstack
command, and also check the output of ipcs -s [-i]. After this has happened,
all new connection will stay in state authentication.