Re: [PATCHES] COPY FROM performance improvements

From: "Luke Lonergan" <llonergan(at)greenplum(dot)com>
To: "Mark Wong" <markw(at)osdl(dot)org>
Cc: "Andrew Dunstan" <andrew(at)dunslane(dot)net>, "Alvaro Herrera" <alvherre(at)surnet(dot)cl>, "Bruce Momjian" <pgman(at)candle(dot)pha(dot)pa(dot)us>, "Alon Goldshuv" <agoldshuv(at)greenplum(dot)com>, pgsql-performance(at)postgresql(dot)org, maryedie(at)osdl(dot)org
Subject: Re: [PATCHES] COPY FROM performance improvements
Date: 2005-07-23 04:09:10
Message-ID: BF0710F6.97DE%llonergan@greenplum.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-patches pgsql-performance

Mark,

On 7/22/05 12:47 PM, "Mark Wong" <markw(at)osdl(dot)org> wrote:

> On a single spindle:
>
> $ time dd if=/dev/zero of=bigfile bs=8k count=2000000
> 2000000+0 records in
> 2000000+0 records out
>
> real 2m8.569s
> user 0m0.725s
> sys 0m19.633s

This is super fast! 124MB/s seems too fast for true write performance on a
single spindle.

> But here are some number for my 12-disk lvm2 striped volume.

So, software striping on how many controllers?

> $ time dd if=/dev/zero of=bigfile3 bs=8k count=4000000
> 4000000+0 records in
> 4000000+0 records out
>
> real 1m17.059s
> user 0m1.479s
> sys 0m41.293s

Again - super fast at 416MB/s. How many controllers?

When we had our problems with the cciss driver and the smartarray 5i/6
controllers, we found the only way to get any performance out of them was to
run them in JBOD mode and software stripe. However, when we did so the CPU
usage skyrocketed and the performance of simple SCSI adapters was 50% faster
with less CPU consumption. These numbers show 100*(42.8/67) = 64% CPU
consumption, I'd expect less with 2 simple U320 SCSI controllers.

- Luke

In response to

Browse pgsql-patches by date

  From Date Subject
Next Message Bruce Momjian 2005-07-23 14:25:42 Re: [HACKERS] Timezone bugs
Previous Message Bruce Momjian 2005-07-23 02:04:50 Re: [HACKERS] Timezone bugs

Browse pgsql-performance by date

  From Date Subject
Next Message Sam Mason 2005-07-23 10:42:35 Re: [HACKERS] Planner doesn't look at LIMIT?
Previous Message Dawid Kuroczko 2005-07-22 20:48:58 Re: Planner doesn't look at LIMIT?