Re: speeding up planning with partitions

From: Amit Langote <amitlangote09(at)gmail(dot)com>
To: Justin Pryzby <pryzby(at)telsasoft(dot)com>
Cc: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, "Imai, Yoshikazu" <imai(dot)yoshikazu(at)jp(dot)fujitsu(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: speeding up planning with partitions
Date: 2019-02-22 17:54:35
Message-ID: CA+HiwqEUwfg4s=4=is-+7k-WM-2Gs9K3iFB96QNm_eafcBozqQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Justin,

Thanks for checking.

On Sat, Feb 23, 2019 at 1:59 AM Justin Pryzby <pryzby(at)telsasoft(dot)com> wrote:
>
> On Fri, Feb 22, 2019 at 09:45:38PM +0900, Amit Langote wrote:
> > I have updated the inheritance expansion patch.
> >
> > Patch 0001 rewrites optimizer/utils/inherit.c, so that it allows
>
> Thanks for your continued work on this.
>
> I applied v23 patch and imported one of our customers' schema, and ran explain
> on a table with 210 partitions. With patch applied there are 10x fewer system
> calls, as intended.
>
> with patch:
> 173 pread64
> 76 lseek
> 47 open
> 38 brk
>
> without patch:
> 1276 lseek
> 693 pread64
> 647 open
> 594 brk

OK, great. I guess you were running SELECT? Just in case you missed,
the patch to improve UPDATE/DELETE scalability is no longer part of
this patch series.

> > + if (IS_SIMPLE_REL(rel1) && child_rel1 == NULL)
> > + child_rel1 = build_dummy_partition_rel(root, rel1, baserel1,
> > + cnt_parts);
> > + if (IS_SIMPLE_REL(rel1) && child_rel2 == NULL)
> > + child_rel2 = build_dummy_partition_rel(root, rel2, baserel2,
> > + cnt_parts);
>
> Should 2nd "if" say IS_SIMPLE_REL(rel2) ?

Good catch, fixed. Apparently not guarded by a test, but I haven't
bothered to add new tests with this patch series.

Please find attached updated patches. I've made a few updates in last
couple of hours such as improving comments, fixing a few thinkos in
inheritance_planner changes, etc.

Thanks,
Amit

Attachment Content-Type Size
v24-0003-Do-not-lock-all-partitions-at-the-beginning.patch application/octet-stream 2.5 KB
v24-0002-Teach-planner-to-only-process-unpruned-partition.patch application/octet-stream 6.9 KB
v24-0001-Lazy-creation-of-RTEs-for-inheritance-children.patch application/octet-stream 91.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2019-02-22 17:54:47 Re: Autovaccuum vs temp tables crash
Previous Message Tom Lane 2019-02-22 17:45:55 Re: boolean and bool in documentation