Re: speeding up planning with partitions

From: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
To: Imai Yoshikazu <yoshikazu_i443(at)live(dot)jp>, "jesper(dot)pedersen(at)redhat(dot)com" <jesper(dot)pedersen(at)redhat(dot)com>, "Imai, Yoshikazu" <imai(dot)yoshikazu(at)jp(dot)fujitsu(dot)com>, "'Amit Langote'" <amitlangote09(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: speeding up planning with partitions
Date: 2019-03-22 07:39:30
Message-ID: c8dcf0fd-4759-5e1c-ea67-e2d3f98e81bd@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Hi,

On 2019/03/22 11:17, Amit Langote wrote:
> However, looking into make_partitionedrel_pruneinfo(), it seems that it's
> unconditionally allocating 3 arrays that all have nparts elements:
>
> subplan_map = (int *) palloc0(nparts * sizeof(int));
> subpart_map = (int *) palloc0(nparts * sizeof(int));
> relid_map = (Oid *) palloc0(nparts * sizeof(int));
>
> So, that part has got to cost more as the partition count grows.
>
> This is the code for runtime pruning, which is not exercised in our tests,
> so it might seem odd that we're spending any time here at all. I've been
> told that we have to perform at least *some* work here if only to conclude
> that runtime pruning is not needed and it seems that above allocations
> occur before making that conclusion. Maybe, that's salvageable by
> rearranging this code a bit. David may be in a better position to help
> with that.

I took a stab at this. I think rearranging the code in
make_partitionedrel_pruneinfo() slightly will take care of this.

The problem is that make_partitionedrel_pruneinfo() does some work which
involves allocating arrays containing nparts elements and then looping
over the part_rels array to fill values in those arrays that will be used
by run-time pruning. It does all that even *before* it has checked that
run-time pruning will be needed at all, which if it is not, it will simply
discard the result of the aforementioned work.

Patch 0008 of 0009 rearranges the code such that we check whether we will
need run-time pruning at all before doing any of the work that's necessary
for run-time to work.

Thanks,
Amit

Attachment Content-Type Size
v34-0001-Build-other-rels-of-appendrel-baserels-in-a-sepa.patch text/plain 11.8 KB
v34-0002-Add-rowmark-junk-targetlist-entries-in-query_pla.patch text/plain 25.6 KB
v34-0003-Delay-adding-inheritance-child-tables-until-quer.patch text/plain 30.4 KB
v34-0004-Adjust-inheritance_planner-to-reuse-source-child.patch text/plain 27.2 KB
v34-0005-Further-tweak-inheritance_planner-to-avoid-usele.patch text/plain 4.7 KB
v34-0006-Perform-pruning-in-expand_partitioned_rtentry.patch text/plain 30.0 KB
v34-0007-Teach-planner-to-only-process-unpruned-partition.patch text/plain 7.6 KB
v34-0008-Rearrange-the-code-in-make_partitionedrel_prunei.patch text/plain 4.7 KB
v34-0009-Don-t-copy-PartitionBoundInfo-in-set_relation_pa.patch text/plain 1.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrey Borodin 2019-03-22 08:00:05 Re: GiST VACUUM
Previous Message Haribabu Kommi 2019-03-22 07:07:16 Re: Libpq support to connect to standby server as priority