RE: speeding up planning with partitions

From: "Imai, Yoshikazu" <imai(dot)yoshikazu(at)jp(dot)fujitsu(dot)com>
To: 'Amit Langote' <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
Cc: 'Amit Langote' <amitlangote09(at)gmail(dot)com>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, David Rowley <david(dot)rowley(at)2ndquadrant(dot)com>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, "jesper(dot)pedersen(at)redhat(dot)com" <jesper(dot)pedersen(at)redhat(dot)com>, Justin Pryzby <pryzby(at)telsasoft(dot)com>, "Pg Hackers" <pgsql-hackers(at)postgresql(dot)org>
Subject: RE: speeding up planning with partitions
Date: 2019-03-08 07:16:30
Message-ID: 0F97FA9ABBDBE54F91744A9B37151A5129CC1D@g01jpexmbkw24
Views: Raw Message | Whole Thread | Download mbox
Thread:
Lists: pgsql-hackers

Amit-san,

On Wed, Mar 6, 2019 at 5:38 AM, Amit Langote wrote:
...
> > I didn't investigate that problem, but there is another memory
> > increase
> issue, which is because of 0003 patch I think. I'll try to solve the latter
> issue.
>
> Interested in details as it seems to be a separate problem.

I solved this problem.

I think we don't need to do list_copy in the below code.

+ /*
+ * No need to copy of the RTEs themselves, but do copy the List
+ * structure.
+ */
+ subroot->parse->rtable = list_copy(rtable_with_target);

Because subroot->parse->rtable will be copied again by:

subroot->parse = (Query *)
adjust_appendrel_attrs(parent_root,
- (Node *) parent_parse,
+ (Node *) subroot->parse,
1, &appinfo);

So I modified the code and did test to confirm memory increasement don't happen. The test and results are below.

[test]
* Create partitioned table with 1536 partitions.
* Execute "update rt set a = random();"

[results]
A backend uses below amount of memory in update transaction:

HEAD: 807MB
With v26-0001, 0002: 790MB
With v26-0001, 0002, 0003: 860MB
With v26-0003 modified: 790MB

I attached the diff of modification for v26-0003 patch which also contains some refactoring.
Please see if it is ok.
(Sorry it is modification for v26 patch though latest ones are v28.)

--
Yoshikazu Imai

Attachment Content-Type Size
v26-0003-solving-memory-increasement-problem.diff application/octet-stream 3.3 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Antonin Houska 2019-03-08 07:36:34 Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)
Previous Message Masahiko Sawada 2019-03-08 06:45:18 Re: [Proposal] Table-level Transparent Data Encryption (TDE) and Key Management Service (KMS)