From: | Richard Broersma Jr <rabroersma(at)yahoo(dot)com> |
---|---|
To: | Neil Aggarwal <neil(at)JAMMConsulting(dot)com>, pgsql-novice(at)postgresql(dot)org |
Subject: | Re: Crappy performance even though not swapping |
Date: | 2007-02-28 14:52:07 |
Message-ID: | 846669.6097.qm@web31809.mail.mud.yahoo.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-novice |
> I have a Java program that is trying to import over 8 million
> rows from a text file.
>
> When the program starts up, it runs fine, but after a while
> it begins to crawl.
>
> This is usually symptomatic of a machine that started swapping,
> but take a look at the attached output from top.
>
> I am using a LOT of memory, but the swap is still very
> low. I don't think the machine should be paging to disk.
>
> The CPU states show that the cpus are spending most of their time
> in an idle state. The process list shows the postmaster and java
> processes are less than 10 percent the CPU cycles.
>
> Having said that, the load average is still about 2.
>
> Any ideas what could be going on?
Your question is a good one for the performance list. From what I've read there, there are a
couple of strategies that you can use to improve performance.
1) if you can roll large numbers of insert statements (say 1K to 10K ins/trans) in to single
transactions, performance should be helped.
2) If this is a one time insert, you may consider TEMPORARILY turning off fsync and greatly
increase your check point segment size for the import.
Regards,
Richard Broersma Jr.
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Wimmer | 2007-02-28 16:16:08 | Re: automatically dump a table when it is updated |
Previous Message | Jim Stalewski | 2007-02-28 14:18:35 | Re: Crappy performance even though not swapping |