Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning

From: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
To: David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>
Cc: Kyotaro HORIGUCHI <horiguchi(dot)kyotaro(at)lab(dot)ntt(dot)co(dot)jp>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Dilip Kumar <dilipbalaut(at)gmail(dot)com>, Beena Emerson <memissemerson(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [Sender Address Forgery]Re: [HACKERS] path toward faster partition pruning
Date: 2018-01-18 10:56:47
Message-ID: 0af7826d-3ee2-0187-f622-f86542a4f61e@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi David.

On 2018/01/18 12:14, David Rowley wrote:
> On 18 January 2018 at 00:13, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com> wrote:
>> On 17 January 2018 at 23:48, Amit Langote <amitlangote09(at)gmail(dot)com> wrote:
>>> I'm concerned that after your patch to remove
>>> match_clauses_to_partkey(), we'd be doing more work than necessary in
>>> some cases. For example, consider the case of using run-time pruning
>>> for nested loop where the inner relation is a partitioned table. With
>>> the old approach, get_partitions_from_clauses() would only be handed
>>> the clauses that are known to match the partition keys (which most
>>> likely is fewer than all of the query's clauses), so
>>> get_partitions_from_clauses() doesn't have to do the work of filtering
>>> non-partition clauses every time (that is, for every outer row).
>>> That's why I had decided to keep that part in the planner.
>>
>> That might be better served by splitting
>> classify_partition_bounding_keys() into separate functions, the first
>> function would be in charge of building keyclauses_all. That way the
>> remaining work during the executor would never need to match clauses
>> to a partition key as they'd be in lists dedicated to each key.
>
> I've attached another delta against your v20 patch which does this.
> It's very rough for now and I've only checked that it passes the
> regression test so far.

Thanks!

> It will need some cleanup work, but I'd be keen to know what you think
> of the general idea.

This one looks in a much better shape.

> I've not fully worked out how run-time pruning
> will use this as it'll need another version of
> get_partitions_from_clauses but passes in a PartScanClauseInfo
> instead, and does not call extract_partition_key_clauses. That area
> probably needs some shuffling around so that does not end up a big
> copy and paste of all that new logic.
So, I've been assuming that the planner changes in the run-time pruning
patch have to do with selecting clauses (restriction clauses not
containing Consts and/or join clauses) to be passed to the executor by
recording them in the Append node. Will they be selected by the planner
calling into partition.c?

Meanwhile, here is a revised version (v21) that incorporates your changes.
I added you as the author in 0002 and 0005 patches. I guess a v22 will
have to follow very soon...

Thanks,
Amit

Attachment Content-Type Size
v21-0001-Some-interface-changes-for-partition_bound_-cmp-.patch text/plain 11.7 KB
v21-0002-Introduce-a-get_partitions_from_clauses.patch text/plain 66.6 KB
v21-0003-Move-some-code-of-set_append_rel_size-to-separat.patch text/plain 8.6 KB
v21-0004-More-refactoring-around-partitioned-table-Append.patch text/plain 12.7 KB
v21-0005-Teach-planner-to-use-get_partitions_from_clauses.patch text/plain 34.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message hubert depesz lubaczewski 2018-01-18 11:04:24 Re: Is there a "right" way to test if a database is empty?
Previous Message Thomas Munro 2018-01-18 10:49:51 Re: [HACKERS] Parallel tuplesort (for parallel B-Tree index creation)