Skip site navigation (1) Skip section navigation (2)

Re: Crappy performance even though not swapping

From: Richard Broersma Jr <rabroersma(at)yahoo(dot)com>
To: Neil Aggarwal <neil(at)JAMMConsulting(dot)com>, pgsql-novice(at)postgresql(dot)org
Subject: Re: Crappy performance even though not swapping
Date: 2007-02-28 14:52:07
Message-ID: 846669.6097.qm@web31809.mail.mud.yahoo.com (view raw or flat)
Thread:
Lists: pgsql-novice
> I have a Java program that is trying to import over 8 million 
> rows from a text file.
> 
> When the program starts up, it runs fine, but after a while
> it begins to crawl.
> 
> This is usually symptomatic of a machine that started swapping,
> but take a look at the attached output from top.
> 
> I am using a LOT of memory, but the swap is still very 
> low.  I don't think the machine should be paging to disk.
> 
> The CPU states show that the cpus are spending most of their time 
> in an idle state.  The process list shows the postmaster and java 
> processes are less than 10 percent the CPU cycles.
> 
> Having said that, the load average is still about 2.
> 
> Any ideas what could be going on?

Your question is a good one for the performance list.  From what I've read there, there are a
couple of strategies that you can use to improve performance.

1) if you can roll large numbers of insert statements (say 1K to 10K ins/trans) in to single
transactions, performance should be helped.

2) If this is a one time insert, you may consider TEMPORARILY turning off fsync and greatly
increase your check point segment size for the import.

Regards,
Richard Broersma Jr.

In response to

pgsql-novice by date

Next:From: Robert WimmerDate: 2007-02-28 16:16:08
Subject: Re: automatically dump a table when it is updated
Previous:From: Jim StalewskiDate: 2007-02-28 14:18:35
Subject: Re: Crappy performance even though not swapping

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group