Re: Re: parallel distinct union and aggregate support patch

From: "bucoo(at)sohu(dot)com" <bucoo(at)sohu(dot)com>
To: "David Steele" <david(at)pgmasters(dot)net>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: tgl <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Dilip Kumar" <dilipbalaut(at)gmail(dot)com>, "Thomas Munro" <thomas(dot)munro(at)gmail(dot)com>, "Tomas Vondra" <tomas(dot)vondra(at)2ndquadrant(dot)com>, hlinnaka <hlinnaka(at)iki(dot)fi>, robertmhaas <robertmhaas(at)gmail(dot)com>, pgsql <pgsql(at)j-davis(dot)com>
Subject: Re: Re: parallel distinct union and aggregate support patch
Date: 2021-03-30 09:32:45
Message-ID: 202103301732449250842@sohu.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> This patch has not gotten any review in the last two CFs and is unlikely
> to be committed for PG14 so I have moved it to the 2021-07 CF. A rebase
> is also required so marked Waiting for Author.
>
> I can see this is a work in progress, but you may want to consider the
> several suggestions that an unbuffered approach might be better.

I have written a plan with similar functions, It is known that the following two situations do not work well.
1. Under "Parallel Append" plan
Gather
-> Parallel Append
-> Agg
-> Parallel Redistribute(1)
-> ...
-> Agg
-> Parallel Redistribute(2)
-> ...
when parallel worker 1 execute "Parallel Redistribute(1)" and worker execute "Parallel Redistribute(2)",
both "Parallel Redistribute" plan can not send tuples to other worker(both worker are stuck),
because outher worker's memory buffer run out soon.

2. Under "Nestloop" plan
Gather
-> Nestloop(1)
-> Nestloop(2)
-> Parallel Redistribute
-> ...
-> IndexScan
-> Agg
At some point might be the case: parallel worker 1 executing Agg and "Parallel Redistribute" plan's memory buffer is full,
worker 2 executing "Parallel Redistribute" and it waiting worker 1 eat "Parallel Redistribute" plan's memory buffer,
it's stuck.

bucoo(at)sohu(dot)com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Markus Wanner 2021-03-30 09:46:37 Re: [PATCH] Provide more information to filter_prepare
Previous Message Bharath Rupireddy 2021-03-30 09:26:42 Re: Use consistent terminology for tablesync slots.