RE: speeding up planning with partitions

From: "Imai, Yoshikazu" <imai(dot)yoshikazu(at)jp(dot)fujitsu(dot)com>
To: "Imai, Yoshikazu" <imai(dot)yoshikazu(at)jp(dot)fujitsu(dot)com>, 'Amit Langote' <amitlangote09(at)gmail(dot)com>
Cc: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, "Alvaro Herrera" <alvherre(at)2ndquadrant(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>, Jesper Pedersen <jesper(dot)pedersen(at)redhat(dot)com>
Subject: RE: speeding up planning with partitions
Date: 2019-03-14 08:35:09
Message-ID: 0F97FA9ABBDBE54F91744A9B37151A5129E34B@g01jpexmbkw24
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Amit-san,

I have done code review of v31 patches from 0004 to 0008.

0004:
* s/childern/children

0005:
* This seems reasonable for not using a lot of memory in specific case, although it needs special looking of planner experts.

0006:
* The codes initializing/setting RelOptInfo's part_rels looks like a bit complicated, but I didn't come up with any good design so far.

0007:
* This changes some processes using "for loop" to using "while(bms_next_member())" which speeds up processing when we scan few partitions in one statement, but when we scan a lot of partitions in one statement, its performance will likely degraded. I measured the performance of both cases.
I executed select statement to the table which has 4096 partitions.

[scanning 1 partition]
Without 0007 : 3,450 TPS
With 0007 : 3,723 TPS

[scanning 4096 partitions]
Without 0007 : 10.8 TPS
With 0007 : 10.5 TPS

In the above result, performance degrades 3% in case of scanning 4096 partitions compared before and after applying 0007 patch. I think when scanning a lot of tables, executor time would be also longer, so the increasement of planner time would be relatively smaller than it. So we might not have to care this performance degradation.

0008:
This seems ok.

--
Yoshikazu Imai

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2019-03-14 08:42:49 Re: Sparse bit set data structure
Previous Message Dean Rasheed 2019-03-14 08:28:16 Re: pgsql: Add support for hyperbolic functions, as well as log10().