Skip site navigation (1) Skip section navigation (2)

LIMIT confuses the planner

From: Kouber Saparev <kouber(at)saparev(dot)com>
To: pgsql-performance(at)postgresql(dot)org
Subject: LIMIT confuses the planner
Date: 2009-02-23 12:26:24
Message-ID: 49A295F0.5010405@saparev.com (view raw or flat)
Thread:
Lists: pgsql-performance
Hello,

I'm experiencing a strange issue. I have a table with around 11 million 
records (11471762 to be exact), storing login attempts to a web site. 
Thanks to the index I have created on username, looking into that table 
by username is very fast:



db=# EXPLAIN ANALYZE
SELECT
   *
FROM
   login_attempt
WHERE
   username='kouber'
ORDER BY
   login_attempt_sid DESC;
 
QUERY PLAN
------------------------------------------------------------------------------------------------------------------------------------------------------
  Sort  (cost=1415.15..1434.93 rows=7914 width=38) (actual 
time=0.103..0.104 rows=2 loops=1)
    Sort Key: login_attempt_sid
    Sort Method:  quicksort  Memory: 25kB
    ->  Index Scan using login_attempt_username_idx on login_attempt 
(cost=0.00..902.71 rows=7914 width=38) (actual time=0.090..0.091 rows=2 
loops=1)
          Index Cond: ((username)::text = 'kouber'::text)
  Total runtime: 0.140 ms
(6 rows)



As you can see, there are only 2 records for that particular username.

However when I add a LIMIT clause to the same query the planner no 
longer uses the right index, hence the query becomes very slow:



db=# EXPLAIN ANALYZE
SELECT
   *
FROM
   login_attempt
WHERE
   username='kouber'
ORDER BY
   login_attempt_sid DESC LIMIT 20;
 
   QUERY PLAN
-------------------------------------------------------------------------------------------------------------------------------------------------------------
  Limit  (cost=0.00..770.45 rows=20 width=38) (actual 
time=0.064..3797.660 rows=2 loops=1)
    ->  Index Scan Backward using login_attempt_pkey on login_attempt 
(cost=0.00..304866.46 rows=7914 width=38) (actual time=0.062..3797.657 
rows=2 loops=1)
          Filter: ((username)::text = 'kouber'::text)
  Total runtime: 3797.691 ms
(4 rows)



Now, recently I have altered some of the default parameters in order to 
get as much as possible out of the hardware - 12 GB of RAM, 8 
processors. So, I guess I have done something wrong, thus the planner is 
taking that wrong decision. Here's what I have changed in 
postgresql.conf (from the default one):

max_connections = 200
shared_buffers = 256MB
work_mem = 64MB
maintenance_work_mem = 128MB
max_stack_depth = 6MB
max_fsm_pages = 100000
synchronous_commit = off
wal_buffers = 1MB
commit_delay = 100
commit_siblings = 5
checkpoint_segments = 10
checkpoint_timeout = 10min
random_page_cost = 0.1
effective_cache_size = 2048MB

Any idea what's wrong here?

Regards,
-- 
Kouber Saparev
http://kouber.saparev.com

Responses

pgsql-performance by date

Next:From: Richard HuxtonDate: 2009-02-23 13:27:44
Subject: Re: LIMIT confuses the planner
Previous:From: Mark WongDate: 2009-02-22 23:03:28
Subject: Re: dbt-2 tuning results with postgresql-8.3.5

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group