Re: Partial aggregates pushdown

From: Alexander Pyhalov <a(dot)pyhalov(at)postgrespro(dot)ru>
To: Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>
Cc: pgsql-hackers(at)postgresql(dot)org
Subject: Re: Partial aggregates pushdown
Date: 2021-10-21 10:55:04
Message-ID: 07fa4d5a56f402732acfce25e7207cae@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Tomas Vondra писал 2021-10-19 16:25:
> On 10/19/21 08:56, Alexander Pyhalov wrote:
>> Hi.
>>
>> Tomas Vondra писал 2021-10-15 17:56:
>>> As for the proposed approach, it's probably good enough for the first
>>> version to restrict this to aggregates where the aggregate result is
>>> sufficient, i.e. we don't need any new export/import procedures.
>>>
>>> But it's very unlikely we'd want to restrict it the way the patch
>>> does
>>> it, i.e. based on aggregate name. That's both fragile (people can
>>> create new aggregates with such name) and against the PostgreSQL
>>> extensibility (people may implement custom aggregates, but won't be
>>> able to benefit from this just because of name).
>>>
>>> So for v0 maybe, but I think there neeeds to be a way to relax this
>>> in
>>> some way, for example we could add a new flag to pg_aggregate to mark
>>> aggregates supporting this.
>>>
>>
>> Updated patch to mark aggregates as pushdown-safe in pg_aggregates.
>>
>> So far have no solution for aggregates with internal aggtranstype.

Hi. Updated patch.
Now aggregates with internal states can be pushed down, if they are
marked as pushdown safe (this flag is set to true for min/max/sum),
have internal states and associated converters. Converters are called
locally, they transform aggregate result to serialized internal
representation.
As converters don't have access to internal aggregate state, partial
aggregates like avg() are still not pushable.

For now the overall logic is quite simple. We now also call
add_foreign_grouping_paths() for partial aggregation. In
foreign_expr_walker() we check if aggregate is pushable (which means
that it is simple, marked as pushable and if has 'internal' as
aggtranstype, has associated converter).
If it is pushable, we proceed as with usual aggregates (but forbid
having pushdown). During postgresGetForeignPlan() we produce list of
converters for aggregates. As converters has different input argument
type from their result (bytea), we have to generate alternative
metadata, which is used by make_tuple_from_result_row().
If make_tuple_from_result_row() encounters field with converter, it
calls converter and returns result. For now we expect converter to have
only one input and output argument. Existing converters just transform
input value to internal representation and return its serialized form.

--
Best regards,
Alexander Pyhalov,
Postgres Professional

Attachment Content-Type Size
0001-Partial-aggregates-push-down-v03.patch text/x-diff 57.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Japin Li 2021-10-21 11:09:13 Re: [Bug] Logical Replication failing if the DateStyle is different in Publisher & Subscriber
Previous Message hubert depesz lubaczewski 2021-10-21 10:52:53 Re: Can we get rid of repeated queries from pg_dump?