Re: best paging strategies for large datasets?

On Thu, May 13, 2010 at 11:36:53AM +0400, silly sad wrote:
> On 05/12/10 09:41, Louis-David Mitterrand wrote:
> >Hi,
> >
> >I have a large dataset (page 1 at http://www.cruisefish.net/stat.md) and
> >am in the process of developping a pager to let users leaf through it
> >(30K rows).
> >
> >Ideally I'd like to know when requesting any 'page' of data where I am
> >within the dataset: how many pages are available each way, etc.
> >
> >Of course that can be done by doing a count(*) query before requesting a
> >limit/offset subset. But the main query is already quite slow, so I'd
> >like to minimize them.
>
> nowadays i tend to bet on AJAX.
> in other words i propose to move some calculations to a client side at all.
>
> and this particular situation might looks similar to the following:
>
> First u count(*) the rows and select a requested page
> returning to a client the count result bundled "with a page of rows"
>
> (1) client renders the acquired rows
> (2)__memorize__ what part of the data he just got
> (3) and stores the count result to calculate "the pager div"
>
> all the subsequent clicks on "the pager div" should not immediately
> generate requests and decides if the request is needed.

Yes, rendering the results throught ajax is a good idea, but one has to
be careful not to expose one's LIMIT and OFFSET to the client, but only
the "page" number. Or else the client could query the whole data set. A
lot of "professional" web site have that hole.