Speeding up creating UPDATE/DELETE generic plan for partitioned table into a lot

From: "Kato, Sho" <kato-sho(at)jp(dot)fujitsu(dot)com>
To: "Tsunakawa, Takayuki" <tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com>, 'David Rowley' <david(dot)rowley(at)2ndquadrant(dot)com>, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Speeding up creating UPDATE/DELETE generic plan for partitioned table into a lot
Date: 2019-02-01 08:16:11
Message-ID: 25C1C6B2E7BE044889E4FE8643A58BA963DEA123@G01JPEXMBKW03
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Sorry, I lost previous mail[1].

On Fri, 28 Dec 2018 at 20:36, Tsunakawa, Takayuki
<tsunakawa(dot)takay(at)jp(dot)fujitsu(dot)com> wrote:
> Although I may say the same thing as you, I think a natural idea would be to create a generic plan gradually. The starting simple question is "why do we have to touch all partitions at first?" That is, can we behave like this:

I also think creating a generic plan gradually is a better idea because planner should create a plan when it is needed.
Any ideas?

On 2018-12-31 08:57:04, David Rowley wrote
>I imagine the place to start looking would be around why planning is so slow for that many partitions.

As you may already know, FLATCOPY at range_table_mutator has a large bottleneck.
Executing UPDATE, about npart squared RangeTblEntry is copied.
When I execute UPDATE to 100 partitioned table, FLATCOPY takes about 100 * 0.067 ms while total planning time takes 12.689 ms.

On 2018-12-31 08:57:04, David Rowley wrote
>Another possible interesting idea would be to, instead of creating
>large Append/MergeAppend plans for partition scanning, invent some
>"Partition Seq Scan" and "Partition Index Scan" nodes that are able to
>build plans more similar to scanning a normal table. Likely such nodes
>would need to be programmed with a list of Oids that they're to scan
>during their execution. They'd also need to take care of their own
>tuple mapping for when partitions had their columns in varying orders.

Inventing some "Partition Seq Scan" and "Partition Index Scan" nodes is interesting.
It seems easy to add Scan nodes to each partition gradually.

[1]:CAKJS1f-y1HQK+VjG7=C==vGcLnzxjN8ysD5NmaN8Wh4=VsYipw(at)mail(dot)gmail(dot)com

regards,

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2019-02-01 08:22:19 Re: current_logfiles not following group access and instead follows log_file_mode permissions
Previous Message Michael Paquier 2019-02-01 08:14:37 Re: initdb --allow-group-access behaviour in windows