Inefficient SELECT with OFFSET and LIMIT

From: Clive Page <cgp(at)leicester(dot)ac(dot)uk>
To: pgsql-performance(at)postgresql(dot)org
Subject: Inefficient SELECT with OFFSET and LIMIT
Date: 2004-01-06 15:40:48
Message-ID: Pine.SGI.4.21.0401061539460.10317357-100000@harrier.le.ac.uk
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

I have just discovered that if one does a SELECT with a LIMIT and OFFSET
values, say

SELECT myfunc(mycol) FROM table LIMIT 50 OFFSET 10000 ;

Then the whole of the selection expressions, including the function calls,
are actuall executed for every record, not just those being selected but
also those being skipped, i.e. 10050 in this case.
Actually it's even odder, as the number is that plus one, as the next
record in sequence is also passed to the function.

I discovered this by accident, since I was using a user-defined function
in pl/pgsql and included by mistake some debug code using RAISE INFO, so
this diagnostic output gave the game away (and all of it came out before
any of the results of the selection, which was another surprise).

It looks as if OFFSET is implemented just be throwing away the results,
until the OFFSET has been reached.

It would be nice if OFFSET could be implemented in some more efficient
way.

--
Clive Page,
Dept of Physics & Astronomy,
University of Leicester, U.K.

Responses

Browse pgsql-performance by date

  From Date Subject
Next Message Robert Treat 2004-01-06 16:03:23 Re: Select max(foo) and select count(*) optimization
Previous Message Shridhar Daithankar 2004-01-06 12:20:09 Re: Select max(foo) and select count(*) optimization