Skip site navigation (1) Skip section navigation (2)

Re: Best COPY Performance

From: "Michael Artz" <mlartz(at)gmail(dot)com>
To: "Luke Lonergan" <llonergan(at)greenplum(dot)com>
Cc: "Worky Workerson" <worky(dot)workerson(at)gmail(dot)com>, "Merlin Moncure" <mmoncure(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Best COPY Performance
Date: 2006-10-28 12:03:10
Message-ID: e9c163070610280503t4788ab5ah857ea249b5027e2@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
> > Are you saying that I should be able to issue multiple COPY commands
> > because my I/O wait is low?  I was under the impression that I am I/O
> > bound, so multiple simeoultaneous loads would have a detrimental
> > effect ...
>
> The reason I asked how many CPUs was to make sense of the 12% usr CPU time
> in the above.  That means you are CPU bound and are fully using one CPU.  So
> you aren't being limited by the I/O in this case, it's the CPU.
>
> I agree with Merlin that you can speed things up by breaking the file up.
> Alternately you can use the OSS Bizgres java loader, which lets you specify
> the number of I/O threads with the "-n" option on a single file.

Thanks, I'll try that on Monday.

> The other thing to wonder about though is why you are so CPU bound at 5
> MB/s.  What version of Postgres is this?

I was wondering about that as well, and the only thing that I can
think of is that its the PK btree index creation on the IP4.

PG 8.1.3 x86_64.  I installed it via a RH rpm for their "Web Services
Beta", or something like that.  I know I'm a bit behind the times, but
getting stuff in (and out) of my isolated lab is a bit of a pain.
I'll compile up a 8.2 beta as well and see how that works out.

In response to

Responses

pgsql-performance by date

Next:From: Luke LonerganDate: 2006-10-28 15:41:55
Subject: Re: Best COPY Performance
Previous:From: Simon RiggsDate: 2006-10-28 10:07:02
Subject: Re: commit so slow program looks frozen

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group