Skip site navigation (1) Skip section navigation (2)

Queries with large ResultSets

From: Jack Orenstein <jao(at)geophile(dot)com>
To: psql-jdbc(at)postgresql(dot)org
Subject: Queries with large ResultSets
Date: 2004-05-19 21:29:00
Message-ID: 40ABD19C.8080109@geophile.com (view raw or flat)
Thread:
Lists: pgsql-jdbc
If I'm reading the source code correctly, the PostgreSQL 7.3.4 JDBC
driver creates a ResultSet by evaluating the entire query, and putting
all the rows into a Vector. ResultSet.next() iterates over the
Vector. Is that in fact what is happening?

If so, this isn't very good for processing large ResultSets. The only
other open-source driver I could find is jxDBCon
(http://jxdbcon.sourceforge.net). Does anyone here have experience
with it?

One obvious application-level workaround is to add ORDER BY <primary
key> and LIMIT to the query, retrieve batches, keeping track of the
last primary key value, etc. That has lots of problems -- changing the
results of the overall scan (since each query runs at a different
time), and doing an index scan instead of a seq scan.

I'm considering modifying the driver, but that would put me behind on
my development schedule.

Jack Orenstein



Responses

pgsql-jdbc by date

Next:From: Oliver JowettDate: 2004-05-20 04:55:14
Subject: Re: Queries with large ResultSets
Previous:From: owDate: 2004-05-19 20:20:42
Subject: Re: [BUG?] Extreme dates

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group