> I have a Java program that is trying to import over 8 million
> rows from a text file.
> When the program starts up, it runs fine, but after a while
> it begins to crawl.
> This is usually symptomatic of a machine that started swapping,
> but take a look at the attached output from top.
> I am using a LOT of memory, but the swap is still very
> low. I don't think the machine should be paging to disk.
> The CPU states show that the cpus are spending most of their time
> in an idle state. The process list shows the postmaster and java
> processes are less than 10 percent the CPU cycles.
> Having said that, the load average is still about 2.
> Any ideas what could be going on?
Your question is a good one for the performance list. From what I've read there, there are a
couple of strategies that you can use to improve performance.
1) if you can roll large numbers of insert statements (say 1K to 10K ins/trans) in to single
transactions, performance should be helped.
2) If this is a one time insert, you may consider TEMPORARILY turning off fsync and greatly
increase your check point segment size for the import.
Richard Broersma Jr.
In response to
pgsql-novice by date
|Next:||From: Robert Wimmer||Date: 2007-02-28 16:16:08|
|Subject: Re: automatically dump a table when it is updated|
|Previous:||From: Jim Stalewski||Date: 2007-02-28 14:18:35|
|Subject: Re: Crappy performance even though not swapping|