Skip site navigation (1) Skip section navigation (2)

Re: Best COPY Performance

From: "Worky Workerson" <worky(dot)workerson(at)gmail(dot)com>
To: "Merlin Moncure" <mmoncure(at)gmail(dot)com>
Cc: "Luke Lonergan" <llonergan(at)greenplum(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Best COPY Performance
Date: 2006-10-28 03:47:06
Message-ID: ce4072df0610272047w4fdf36dfj130b0e9171374d3a@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
On 10/27/06, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> > > r b   swpd  free   buff    cache si so bi    bo   in   cs us sy id wa
> > > 1 0 345732 29328 770980 12947212  0  0 20 16552 1223 3677 12  2 85  1
> > > 1 0 345732 29840 770520 12946924  0  0 20 29244 1283 2955 11  2 85  1
> > > 1 0 345732 32144 770560 12944436  0  0 12 16436 1204 2936 11  2 86  1
> > > 1 0 345732 33744 770464 12942764  0  0 20 16460 1189 2005 10  2 86  1
> > > 2 0 345732 32656 770140 12943972  0  0 16  7068 1057 3434 13  2 85  0
> > > 1 0 345732 34832 770184 12941820  0  0 20  9368 1170 3120 11  2 86  1
> > > 1 0 345732 36528 770228 12939804  0  0 16 32668 1297 2109 11  2 85  1
> > > 1 0 345732 29304 770272 12946764  0  0 16 16428 1192 3105 12  2 85  1
> > > 1 0 345732 30840 770060 12945480  0  0 20 16456 1196 3151 12  2 84  1
> > > 1 0 345732 32760 769972 12943528  0  0 12 16460 1185 3103 11  2 86  1
> >
> > It doesn't look like there's anything else running - the runnable "r" is
> > about 1.  Your "bo" blocks output rate is about 16MB/s, so divide by 3 and
> > you're about in range with your 5MB/s COPY rate.  The interesting thing is
> > that the I/O wait is pretty low.
> >
> > How many CPUs on the machine?  Can you send the result of "cat
> > /proc/cpuinfo"?
>
> iirc, he is running quad opteron 885 (8 cores), so if my math is
> correct he can split up his process for an easy gain.

Are you saying that I should be able to issue multiple COPY commands
because my I/O wait is low?  I was under the impression that I am I/O
bound, so multiple simeoultaneous loads would have a detrimental
effect ...

In response to

Responses

pgsql-performance by date

Next:From: Luke LonerganDate: 2006-10-28 04:07:18
Subject: Re: Best COPY Performance
Previous:From: Worky WorkersonDate: 2006-10-28 03:43:25
Subject: Re: Best COPY Performance

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group