System load consideration before spawning parallel workers

From: Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>
To: "pgsql-hackers(at)postgresql(dot)org" <pgsql-hackers(at)postgresql(dot)org>
Subject: System load consideration before spawning parallel workers
Date: 2016-07-29 05:56:30
Message-ID: CAJrrPGe23T0TQ3V5GABDv0rGdBXhQtCp=SpfY6fJ+YHxuSt5nQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

we observed that spawning the specified number of parallel workers for
every query that satisfies for parallelism is sometimes leading to
performance drop compared to improvement during the peak system load
with other processes. Adding more processes to the system is leading
to more context switches thus it reducing the performance of other SQL
operations.

In order to avoid this problem, how about adding some kind of system
load consideration into account before spawning the parallel workers?

This may not be a problem for some users, so instead of adding the
code into the core for the system load calculation and etc, how about
providing some additional hook in the code? so that user who wants to
consider the system load registers the function and this hooks
provides the number of parallel workers that can be started.

comments?

Regards,
Hari Babu
Fujitsu Australia

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2016-07-29 07:54:42 Re: WAL logging problem in 9.4.3?
Previous Message Kyotaro HORIGUCHI 2016-07-29 05:55:12 Re: Wrong defeinition of pq_putmessage_noblock since 9.5