Re: coypu: "FATAL: sorry, too many clients already"

Thomas Munro <thomas(dot)munro(at)gmail(dot)com> writes:
> Several times on master[1] beginning with an initial occurrence 36
> days ago, and every time on REL_12_STABLE[2], but not on older
> branches, build farm animal coypu has failed in the regression tests
> with the error given in the subject. How can there be too many if
> there are only 20 in a parallel group?

Well, (a) according to the initdb steps in successful runs, coypu
is running with very resource-starved settings, because initdb picks

selecting default max_connections ... 20
selecting default shared_buffers ... 128MB

meaning there is exactly no headroom above the 20 parallel tests
that we sometimes try to launch. (Digging in the buildfarm logs
shows that it used to pick 30, so something's changed there.)

(b) coypu is running with force_parallel_mode = regress, which
comes close to doubling its requirement for backend processes.
(It didn't use to do that, either.)

(c) session disconnect is asynchronous, so previous test backends might
still be cleaning up when new ones try to launch.

The wonder is not that coypu sometimes fails but that it ever succeeds.

I don't see a really good reason to be using force_parallel_mode on
such a low-end box, and would recommend taking that out in any case.
If the box's SysV IPC limits can't be raised, it might be a good idea
to restrict the maximum test parallelism. For instance, I run
prairiedog with

'build_env' => {
'MAX_CONNECTIONS' => '3',
},

because I determined a long time ago that it got through the parallel
tests the fastest that way. (Perhaps this setting is no longer optimal,
but the exact value is not very relevant here.)

regards, tom lane