Skip site navigation (1) Skip section navigation (2)

Re: Best COPY Performance

From: Markus Schaber <schabi(at)logix-tt(dot)com>
To: Worky Workerson <worky(dot)workerson(at)gmail(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: Best COPY Performance
Date: 2006-10-23 09:27:41
Message-ID: 453C8B0D.4070803@logix-tt.com (view raw or flat)
Thread:
Lists: pgsql-performance
Hi, Worky,

Worky Workerson wrote:
> I am currently getting
> between 10K and 15K inserts/second. 

> I ran trivial little insert into a table with a single integer row and
> came close to 250K inserts/second using psql's \copy, so I'm thinking
> that my code could be optimized a bit more, but wanted to check around
> to see if that was the case.

Could you COPY one of your tables out to disk via psql, and then COPY it
back into the database, to reproduce this measurement with your real data?

Also, how much is the disk load, and CPU usage?

As long as psql is factor 20 better than your perl script, I think that
the perl interface is what should be optimized.

On a table with no indices, triggers and contstraints, we managed to
COPY about 7-8 megabytes/second with psql over our 100 MBit network, so
here the network was the bottleneck.

You should think about making your perl program writing the COPY
statement as text, and piping it into psql.

HTH,
Markus
-- 
Markus Schaber | Logical Tracking&Tracing International AG
Dipl. Inf.     | Software Development GIS

Fight against software patents in Europe! www.ffii.org
www.nosoftwarepatents.org

In response to

Responses

pgsql-performance by date

Next:From: John PhilipsDate: 2006-10-23 12:16:37
Subject: Re: Optimizing disk throughput on quad Opteron
Previous:From: Richard HuxtonDate: 2006-10-23 08:45:59
Subject: Re: Vacuum and Memory Loss

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group