Re: [HACKERS] Partition-wise aggregation/grouping

On 15.11.2017 13:35, Jeevan Chalke wrote:
>
> As explained by Ashutosh Bapat in reply
> https://www.postgresql.org/message-id/CAFjFpRdpeMTd8kYbM_x0769V-aEKst5Nkg3+coG=8ki7s8Zqjw@mail.gmail.com
> we cannot rely on just aggtype==aggtranstype.

Obviously this check (aggtype==aggtranstype) is not correct criteria for
all user defined aggregates.
I just did it as temporary work around for standard aggregates.

>
> However, I have tried pushing partial aggregation over remote server
> and also
> submitted a PoC patch here:
> https://www.postgresql.org/message-id/CAM2+6=UakP9+TSJuh2fbhHWNJc7OYFL1_gvu7mt2fXtVt6GY3g@mail.gmail.com
>
> I have later removed these patches from Partition-wise-Aggregation
> patch set
> as it is altogether a different issue than this mail thread. We might
> need to
> discuss on it separately.
...
> Interesting idea of "asynchronous append". However, IMHO it deserves
> its own
> email-chain.

The main problem IMHO is that there are a lot of different threads and
patches related with this topic:(
And it is very difficult to combine all of them together to achieve the
final goal: efficient execution of OLAP queries on sharded table.
It will be nice if somebody who is making the most contribution in this
direction can somehow maintain it...
I just faced with particular problem with our pg_shardman extension and
now (thanks to your patch) I have some working solution for it.
But certainly I prefer to have this support in mainstream version of
Postgres.

There are two open questions, which I wan to discuss (sorry, may be one
again this is not the right thread for it):

1. Parallel append and FDW/postgres_fdw: should FDW support parallel
scan and do we really need it to support concurrent execution of query
on local and remote partitions?
"Asynchronous append" partly solves this problem, but only for remote
partitions. I do not completely understand all complexity of alternative
approaches.

2. Right now partition-wise aggregation/grouping works only for tables
partitioned using new PG 10 partitioning mechanism. But it doesn't work
for inherited tables, although
there seems to be not so much difference between this two cases. Do you
think that sometimes it will be also supported for standard inheritance
mechanism or there is no sense in it?

--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company