Re: System load consideration before spawning parallel workers

On Fri, Aug 5, 2016 at 9:46 AM, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com> wrote:
> On 8/1/16 1:08 AM, Haribabu Kommi wrote:
>>
>> There are some utilities and functions that are available to calculate the
>> current system load, based on the available resources and system load,
>> the module can allow the number of parallel workers that can start. In my
>> observation, adding this calculation will add some overhead for simple
>> queries. Because of this reason, i feel this can be hook function, only
>> for
>> the users who want it, can be loaded.
>
>
> I think we need to provide more tools to allow users to control system
> behavior on a more dynamic basis. How many workers to launch is a good
> example. There's more reasons than just CPU that parallel workers can help
> (IO being an obvious one, but possible other things like GPU). Another
> example is allowing users to alter the selection process used by autovac
> workers.

Yes, we need to consider many parameters as a system load, not just only
the CPU. Here I attached a POC patch that implements the CPU load
calculation and decide the number of workers based on the available CPU
load. The load calculation code is not an optimized one, there are many ways
that can used to calculate the system load. This is just for an example.

Regards,
Hari Babu
Fujitsu Australia