Re: Partition-wise join for join between (declaratively) partitioned tables

From: Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Rafia Sabih <rafia(dot)sabih(at)enterprisedb(dot)com>, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Partition-wise join for join between (declaratively) partitioned tables
Date: 2017-08-10 03:58:40
Message-ID: CAEepm=0VCrC-WfzZkq3YSvJXf225rDnp1ypjv+rjKO5d0=XqFg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

On Thu, Aug 10, 2017 at 1:39 AM, Thomas Munro
<thomas(dot)munro(at)enterprisedb(dot)com> wrote:
> On my computer it took ~1.5 seconds to plan a 1000 partition join,
> ~7.1 seconds to plan a 2000 partition join, and ~50 seconds to plan a
> 4000 partition join. I poked around in a profiler a bit and saw that
> for the 2000 partition case I spent almost half the time in
> create_plan->...->prepare_sort_from_pathkeys->find_ec_member_for_tle,
> and about half of that was in bms_is_subset. The other half the time
> was in query_planner->make_one_rel which spent 2/3 of its time in
> set_rel_size->add_child_rel_equivalences->bms_overlap and the other
> 1/3 in standard_join_search.

Ashutosh asked me how I did that. Please see attached. I was
explaining simple joins like SELECT * FROM foofoo JOIN barbar USING
(a, b). Here also is the experimental hack I tried when I saw
bitmapset.c eating my CPU.

--
Thomas Munro
http://www.enterprisedb.com

Attachment Content-Type Size
bitmapset-track-leading-empty-space.patch application/octet-stream 4.3 KB
make-partitions.sh application/x-sh 2.0 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Masahiko Sawada 2017-08-10 04:15:28 Comment in snapbuild.c file
Previous Message Masahiko Sawada 2017-08-10 03:43:07 Re: Small code improvement for btree