Skip site navigation (1) Skip section navigation (2)

Re: SELECT INTO large FKyed table is slow

From: Mark Kirkwood <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>
To: pgsql-performance(at)postgresql(dot)org
Subject: Re: SELECT INTO large FKyed table is slow
Date: 2010-11-29 07:11:39
Message-ID: 4CF3522B.2050203@catalyst.net.nz (view raw or flat)
Thread:
Lists: pgsql-performance
On 29/11/10 00:46, Mario Splivalo wrote:
>
> This is the slow part:
> INSERT INTO drones_history (sample_id, drone_id, drone_log_notice, 
> drone_temperature, drone_pressure)
> SELECT * FROM tmpUpdate;
>
> For 100 rows this takes around 2 seconds. For 1000 rows this takes 
> around 40 seconds. For 5000 rows this takes around 5 minutes.
> For 50k rows this takes around 30 minutes! Now this is where I start 
> lag because I get new CSV every 10 minutes or so.

Have you created indexes on drones_history(sample_id) and  
drones_history(drone_id)? That would probably help speed up your INSERT 
quite a bit if you have not done so.

Also it would be worthwhile for you to post the output of:

EXPLAIN ANALYZE INSERT INTO drones_history (sample_id, drone_id, 
drone_log_notice, drone_temperature, drone_pressure)
SELECT * FROM tmpUpdate;

to the list, so we can see what is taking the time.

Cheers

Mark

In response to

Responses

pgsql-performance by date

Next:From: Mario SplivaloDate: 2010-11-29 12:23:51
Subject: Re: SELECT INTO large FKyed table is slow
Previous:From: AI RummanDate: 2010-11-29 07:00:40
Subject: Full Text index is not using during OR operation

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group