Re: COPY from question

From: Kevin Brown <kevin(at)sysexperts(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: COPY from question
Date: 2004-02-03 10:57:46
Message-ID: 20040203105746.GE2608@filer
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-performance

Slavisa Garic wrote:
> Using pg module in python I am trying to run the COPY command to populate
> the large table. I am using this to replace the INSERT which takes about
> few hours to add 70000 entries where copy takes minute and a half.

That difference in speed seems quite large. Too large. Are you batching
your INSERTs into transactions (you should be in order to get good
performance)? Do you have a ton of indexes on the table? Does it have
triggers on it or some other thing (if so then COPY may well wind up doing
the wrong thing since the triggers won't fire for the rows it inserts)?

I don't know what kind of schema you're using, but it takes perhaps a
couple of hours to insert 2.5 million rows on my system. But the rows
in my schema may be much smaller than yours.

--
Kevin Brown kevin(at)sysexperts(dot)com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Andrew Dunstan 2004-02-03 14:00:43 session IDs
Previous Message Kevin Brown 2004-02-03 10:38:42 Re: [pgsql-hackers-win32] Sync vs. fsync during checkpoint

Browse pgsql-performance by date

  From Date Subject
Next Message David Teran 2004-02-03 12:54:17 cache whole data in RAM
Previous Message Czuczy Gergely 2004-02-03 07:25:38 PQexecParams and types