From: | "bucoo(at)sohu(dot)com" <bucoo(at)sohu(dot)com> |
---|---|
To: | "David Steele" <david(at)pgmasters(dot)net>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Cc: | tgl <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Dilip Kumar" <dilipbalaut(at)gmail(dot)com>, "Thomas Munro" <thomas(dot)munro(at)gmail(dot)com>, "Tomas Vondra" <tomas(dot)vondra(at)2ndquadrant(dot)com>, hlinnaka <hlinnaka(at)iki(dot)fi>, robertmhaas <robertmhaas(at)gmail(dot)com>, pgsql <pgsql(at)j-davis(dot)com> |
Subject: | Re: Re: parallel distinct union and aggregate support patch |
Date: | 2021-03-30 09:32:45 |
Message-ID: | 202103301732449250842@sohu.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
> This patch has not gotten any review in the last two CFs and is unlikely
> to be committed for PG14 so I have moved it to the 2021-07 CF. A rebase
> is also required so marked Waiting for Author.
>
> I can see this is a work in progress, but you may want to consider the
> several suggestions that an unbuffered approach might be better.
I have written a plan with similar functions, It is known that the following two situations do not work well.
1. Under "Parallel Append" plan
Gather
-> Parallel Append
-> Agg
-> Parallel Redistribute(1)
-> ...
-> Agg
-> Parallel Redistribute(2)
-> ...
when parallel worker 1 execute "Parallel Redistribute(1)" and worker execute "Parallel Redistribute(2)",
both "Parallel Redistribute" plan can not send tuples to other worker(both worker are stuck),
because outher worker's memory buffer run out soon.
2. Under "Nestloop" plan
Gather
-> Nestloop(1)
-> Nestloop(2)
-> Parallel Redistribute
-> ...
-> IndexScan
-> Agg
At some point might be the case: parallel worker 1 executing Agg and "Parallel Redistribute" plan's memory buffer is full,
worker 2 executing "Parallel Redistribute" and it waiting worker 1 eat "Parallel Redistribute" plan's memory buffer,
it's stuck.
bucoo(at)sohu(dot)com
From | Date | Subject | |
---|---|---|---|
Next Message | Markus Wanner | 2021-03-30 09:46:37 | Re: [PATCH] Provide more information to filter_prepare |
Previous Message | Bharath Rupireddy | 2021-03-30 09:26:42 | Re: Use consistent terminology for tablesync slots. |