Forcing cursor behavior

From: Shawn Heisey <pgsql(at)elyograg(dot)org>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: Forcing cursor behavior
Date: 2009-07-21 02:14:36
Message-ID: 4A65248C.8040406@elyograg.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

I have been scouring everything I can find trying to get my problem
solved. I've turned up a number of discussions, but nothing so far that
does what I want.

I've got a java application, to which I cannot get source code, that
needs to process a query with two million rows in it, out of 35 million
total. Currently we use it with another database, and we are trying to
migrate to Postgres. I've got it talking to the database via your
driver, but it runs out of memory because the driver works as designed
and tries to return the entire query at once. If I could modify the
program, then I could just turn off autocommit, set the fetchsize, and
everything would be well, but that's not an option.

I want to modify the driver to force the cursor behavior with a
fetchsize of 10,000 rows. The program does not modify the database, so
I don't need to worry about autocommit. I've tried changing the code in
every place that I can find, but haven't had any luck. It still
compiles and works, but exactly the same way it did before. I have some
experience with code, but my work experience is admin, not programming.
I have no specific experience with Java.

Would it be possible to patch the jdbc driver so that it will internally
turn autcommit off, set the fetchsize to 10,000, and ignore any attempt
by the application to change these parameters? Could someone create
such a patch for me? I am using java 1.5, so the driver compiles as the
3g version.

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Віталій Тимчишин 2009-07-21 04:42:12 Re: Forcing cursor behavior
Previous Message Oliver Jowett 2009-07-18 14:25:20 Re: Speed improvement - Group batch Insert - Rewrite the INSERT at the driver level (using a parameter)