From: | Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com> |
---|---|
To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
Cc: | Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Parallel Append implementation |
Date: | 2017-02-16 04:15:14 |
Message-ID: | CAFjFpRdHfYowszq9nr-339JKK1taeWCB=1d1JObG4tW7W9AdTA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Wed, Feb 15, 2017 at 6:40 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Wed, Feb 15, 2017 at 4:43 AM, Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com> wrote:
>>> On 14 February 2017 at 22:35, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>>>> For example, suppose that I have a scan of two children, one
>>>> of which has parallel_workers of 4, and the other of which has
>>>> parallel_workers of 3. If I pick parallel_workers of 7 for the
>>>> Parallel Append, that's probably too high.
>>
>> In the patch, in such case, 7 workers are indeed selected for Parallel
>> Append path, so that both the subplans are able to execute in parallel
>> with their full worker capacity. Are you suggesting that we should not
>> ?
>
> Absolutely. I think that's going to be way too many workers. Imagine
> that there are 100 child tables and each one is big enough to qualify
> for 2 or 3 workers. No matter what value the user has selected for
> max_parallel_workers_per_gather, they should not get a scan involving
> 200 workers.
If the user is ready throw 200 workers and if the subplans can use
them to speed up the query 200 times (obviously I am exaggerating),
why not to use those? When the user set
max_parallel_workers_per_gather to that high a number, he meant it to
be used by a gather, and that's what we should be doing.
>
> What I was thinking about is something like this:
>
> 1. First, take the maximum parallel_workers value from among all the children.
>
> 2. Second, compute log2(num_children)+1 and round up. So, for 1
> child, 1; for 2 children, 2; for 3-4 children, 3; for 5-8 children, 4;
> for 9-16 children, 5, and so on.
Can you please explain the rationale behind this maths?
>
> 3. Use as the number of parallel workers for the children the maximum
> of the value computed in step 1 and the value computed in step 2.
>
> With this approach, a plan with 100 children qualifies for 8 parallel
> workers (unless one of the children individually qualifies for some
> larger number, or unless max_parallel_workers_per_gather is set to a
> smaller value). That seems fairly reasonable to me.
--
Best Wishes,
Ashutosh Bapat
EnterpriseDB Corporation
The Postgres Database Company
From | Date | Subject | |
---|---|---|---|
Next Message | Joel Jacobson | 2017-02-16 04:26:27 | case_preservation_and_insensitivity = on |
Previous Message | Michael Paquier | 2017-02-16 03:04:32 | Re: SCRAM authentication, take three |