Queries with large ResultSets

From: Jack Orenstein <jao(at)geophile(dot)com>
To: psql-jdbc(at)postgresql(dot)org
Subject: Queries with large ResultSets
Date: 2004-05-19 21:29:00
Message-ID: 40ABD19C.8080109@geophile.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

If I'm reading the source code correctly, the PostgreSQL 7.3.4 JDBC
driver creates a ResultSet by evaluating the entire query, and putting
all the rows into a Vector. ResultSet.next() iterates over the
Vector. Is that in fact what is happening?

If so, this isn't very good for processing large ResultSets. The only
other open-source driver I could find is jxDBCon
(http://jxdbcon.sourceforge.net). Does anyone here have experience
with it?

One obvious application-level workaround is to add ORDER BY <primary
key> and LIMIT to the query, retrieve batches, keeping track of the
last primary key value, etc. That has lots of problems -- changing the
results of the overall scan (since each query runs at a different
time), and doing an index scan instead of a seq scan.

I'm considering modifying the driver, but that would put me behind on
my development schedule.

Jack Orenstein

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Oliver Jowett 2004-05-20 04:55:14 Re: Queries with large ResultSets
Previous Message ow 2004-05-19 20:20:42 Re: [BUG?] Extreme dates