Re: Performance under contention

On 11/22/2010 11:38 PM, Ivan Voras wrote:
> On 11/22/10 16:26, Kevin Grittner wrote:
>> Ivan Voras<ivoras(at)freebsd(dot)org> wrote:
>>> On 11/22/10 02:47, Kevin Grittner wrote:
>>>> Ivan Voras wrote:
>>>>
>>>>> After 16 clients (which is still good since there are only 12
>>>>> "real" cores in the system), the performance drops sharply
>>>>
>>>> Yet another data point to confirm the importance of connection
>>>> pooling. :-)
>>>
>>> I agree, connection pooling will get rid of the symptom. But not
>>> the underlying problem. I'm not saying that having 1000s of
>>> connections to the database is a particularly good design, only
>>> that there shouldn't be a sharp decline in performance when it
>>> does happen. Ideally, the performance should remain the same as it
>>> was at its peek.
>>
>> Well, I suggested that we add an admission control[1] mechanism,
>
> It looks like a hack (and one which is already implemented by connection
> pool software); the underlying problem should be addressed.

My (poor) understanding is that addressing the underlying problem would
require a massive restructure of postgresql to separate "connection and
session state" from "executor and backend". Idle connections wouldn't
require a backend to sit around unused but participating in all-backends
synchronization and signalling. Active connections over a configured
maximum concurrency limit would queue for access to a backend rather
than fighting it out for resources at the OS level.

The trouble is that this would be an *enormous* rewrite of the codebase,
and would still only solve part of the problem. See the prior discussion
on in-server connection pooling and admission control.

Personally I think the current approach is clearly difficult for many
admins to understand and it's unfortunate that it requires external
software to be effective. OTOH, I'm not sure what the answer is.

--
Craig Ringer