Re: parallel distinct union and aggregate support patch

From: David Steele <david(at)pgmasters(dot)net>
To: "bucoo(at)sohu(dot)com" <bucoo(at)sohu(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: tgl <tgl(at)sss(dot)pgh(dot)pa(dot)us>, dilipbalaut <dilipbalaut(at)gmail(dot)com>, "thomas(dot)munro" <thomas(dot)munro(at)gmail(dot)com>, "tomas(dot)vondra" <tomas(dot)vondra(at)2ndquadrant(dot)com>, hlinnaka <hlinnaka(at)iki(dot)fi>, robertmhaas <robertmhaas(at)gmail(dot)com>, pgsql <pgsql(at)j-davis(dot)com>
Subject: Re: parallel distinct union and aggregate support patch
Date: 2021-03-29 13:36:30
Message-ID: 353b3570-7c04-08e0-d0cc-5386dad9c149@pgmasters.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 1/25/21 9:14 AM, bucoo(at)sohu(dot)com wrote:
> Now, I rewrite batch hashagg and sort, add some comment and combin too
> patches. base on master 2ad78a87f018260d4474eee63187e1cc73c9b976.
> They are support rescan and change GUC
> enable_batch_hashagg/enable_batch_sort to
> max_hashagg_batches/max_sort_batch, default value is "0"(mean is disable).
> The "max_hashagg_batches" in grouping sets each chain using this value,
> maybe we need a better algorithm.
> Do not set "max_sort_batch" too large, because each tuplesort's work
> memory is "work_mem/max_sort_batch".
>
> Next step I want use batch sort add parallel merge join(thinks Dilip
> Kumar) and except/intersect support after this patch commit, welcome to
> discuss.

This patch has not gotten any review in the last two CFs and is unlikely
to be committed for PG14 so I have moved it to the 2021-07 CF. A rebase
is also required so marked Waiting for Author.

I can see this is a work in progress, but you may want to consider the
several suggestions that an unbuffered approach might be better.

Regards,
--
-David
david(at)pgmasters(dot)net

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2021-03-29 13:37:58 Re: [PATCH] Provide more information to filter_prepare
Previous Message Ashutosh Bapat 2021-03-29 13:26:59 Re: UniqueKey on Partitioned table.