|From:||Antonin Houska <ah(at)cybertec(dot)at>|
|Subject:||WIP: Aggregation push-down|
|Views:||Raw Message | Whole Thread | Download mbox|
This is a new version of the patch I presented in . A new thread seems
appropriate because the current version can aggregate both base relations and
joins, so the original subject would no longer match.
There's still work to do but I'd consider the patch complete in terms of
concept. A few things worth attention for those who want to look into the
* I've abandoned the concept of aggmultifn proposed in , as it doesn't
appear to be very useful. That implies that a "grouped join" can be formed
in 2 ways: 1) join a grouped relation to a "plain" (i.e. non-grouped) one,
2) join 2 plain relations and aggregate the result. However, w/o the
aggmultifn we can't join 2 grouped relations.
* GroupedVar type is used to propagate the result of partial aggregation from
to the top-level join. It's conceptually very similar to PlaceHolderVar.
* Although I intended to use the "unique join" feature , I postponed it so
far. The point is that  does conflict with my patch and thus I'd have to
rebase the patch more often. Anyway, the impact of  on aggregation
finalization (i.e. possible avoidance of the "finalize aggregate node"
setup) is not really specific to my patch.
* Scan of base relation or join result can be partially aggregated for 2
reasons: 1) it makes the whole plan cheaper because the aggregation takes
place on remote node and thus the amount of data to be transferred via
network is significanlty reduced, 2) aggregate functions are rather
expensive so it makes sense to evaluate them by multiple parallel workers.
The patch contains both of these features as they are hard to
separate from each other.
While 1) needs additional work on postgres_fdw, scripts to simulate 2) are
attached. Planner settings are such that cost of expression evaluation is
significant, so that it's worth to engage multiple parallel workers.
In my environment it yields the following output:
Parallel Finalize HashAggregate
Group Key: a.i
-> Gather Merge
Workers Planned: 4
-> Merge Join
Merge Cond: (b.parent = a.i)
Sort Key: b.parent
-> Parallel Partial HashAggregate
Group Key: b.parent
-> Hash Join
Hash Cond: ((b.parent = c.parent) AND (b.j = c.k))
-> Parallel Seq Scan on b
-> Seq Scan on c
Sort Key: a.i
-> Seq Scan on a
Feedback is appreciated.
|Next Message||Amit Langote||2017-04-04 08:57:54||Re: ANALYZE command progress checker|
|Previous Message||Pavel Stehule||2017-04-04 08:09:20||Re: Variable substitution in psql backtick expansion|