Out of memory error on huge resultset

From: "Nick Fankhauser" <nickf(at)ontko(dot)com>
To: "pgsql-jdbc" <pgsql-jdbc(at)postgresql(dot)org>
Subject: Out of memory error on huge resultset
Date: 2002-10-10 15:24:49
Message-ID: NEBBLAAHGLEEPCGOBHDGCEFIFPAA.nickf@ontko.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general pgsql-hackers pgsql-jdbc


I'm selecting a huge ResultSet from our database- about one million rows,
with one of the fields being varchar(500). I get an out of memory error from
java.

If the whole ResultSet gets stashed in memory, this isn't really surprising,
but I'm wondering why this happens (if it does), rather than a subset around
the current record being cached and other rows being retrieved as needed.

If it turns out that there are good reasons for it to all be in memory, then
my question is whether there is a better approach that people typically use
in this situation. For now, I'm simply breaking up the select into smaller
chunks, but that approach won't be satisfactory in the long run.

Thanks

-Nick

--------------------------------------------------------------------------
Nick Fankhauser nickf(at)ontko(dot)com Phone 1.765.935.4283 Fax 1.765.962.9788
Ray Ontko & Co. Software Consulting Services http://www.ontko.com/

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Michael Kichanov 2002-10-10 15:28:59 Re: connection watchdog
Previous Message Roberto (SmartBit) 2002-10-10 15:17:09 Fw: Getting tables, procedures, etc.

Browse pgsql-hackers by date

  From Date Subject
Next Message Dave Cramer 2002-10-10 15:35:48 Re: Out of memory error on huge resultset
Previous Message Bruce Momjian 2002-10-10 14:59:31 Re: inline newNode()

Browse pgsql-jdbc by date

  From Date Subject
Next Message Dave Cramer 2002-10-10 15:33:07 Re: Getting a ResultSet for a refcursor element.
Previous Message Nic Ferrier 2002-10-09 21:56:07 Re: Getting a ResultSet for a refcursor element.