Re: non-bulk inserts and tuple routing

From: Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: non-bulk inserts and tuple routing
Date: 2018-01-24 08:25:08
Message-ID: 56c9f77b-830c-4652-233e-a1c5fa5a9938@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2018/01/20 7:07, Robert Haas wrote:
> On Fri, Jan 19, 2018 at 3:56 AM, Amit Langote
> <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>> I rebased the patches, since they started conflicting with a recently
>> committed patch [1].
>
> I think that my latest commit has managed to break this pretty thoroughly.

I rebased it. Here are the performance numbers again.

* Uses following hash-partitioned table:

create table t1 (a int, b int) partition by hash (a);
create table t1_x partition of t1 for values with (modulus M, remainder R)
...

* Non-bulk insert uses the following code (insert 100,000 rows one-by-one):

do $$
begin
for i in 1..100000 loop
insert into t1 values (i, i+1);
end loop;
end; $$;

Times in milliseconds:

#parts HEAD Patched
8 6148.313 4938.775
16 8882.420 6203.911
32 14251.072 8595.068
64 24465.691 13718.161
128 45099.435 23898.026
256 87307.332 44428.126

* Bulk-inserting 100,000 rows using COPY:

copy t1 from '/tmp/t1.csv' csv;

Times in milliseconds:

#parts HEAD Patched

8 466.170 446.865
16 445.341 444.990
32 443.544 487.713
64 460.579 435.412
128 469.953 422.403
256 463.592 431.118

Thanks,
Amit

Attachment Content-Type Size
v3-0001-Teach-CopyFrom-to-use-ModifyTableState-for-tuple-.patch text/plain 3.5 KB
v3-0002-ExecFindPartition-refactoring.patch text/plain 4.4 KB
v3-0003-During-tuple-routing-initialize-per-partition-obj.patch text/plain 30.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Daniel Gustafsson 2018-01-24 08:47:50 Re: [HACKERS] Refactoring identifier checks to consistently use strcmp
Previous Message Thomas Munro 2018-01-24 08:23:52 Re: Help needed in using 'on_dsm_detach' callback