OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB

From: Ankit Kumar <ankitk(at)xebia(dot)com>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB
Date: 2009-12-22 04:15:35
Message-ID: 4B3047E7.9020901@xebia.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

Hi

I am running a Criteria.scroll() on postgresql on a DB containing 2M
records. The memory keeps increasing and finally it generates an
OutOfMemoryException. Please can you advice how to fix this.

*Postgresql DB version:* 8.4
*Postgresql Driver Used:* postgresql-8.4-701.jdbc4.jar

Some forums do mention that "/scroll/" is slow on Postgresql.

Is this a known issue or we need to do something specific to make scroll
work on Postgresql?

It appears the driver class is loading all entity objects in memory and
not releasing them as expected in a scroll operation.

Appreciate any guidance/suggestion.

*Exception StackTrace*:
java.lang.OutOfMemoryError: Java heap space
at java.lang.Class.getDeclaredFields0(Native Method)
at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
at java.lang.Class.getDeclaredField(Class.java:1880)
at
java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(AtomicReferenceFieldUpdater.java:181)
at
java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:65)
at java.sql.SQLException.<clinit>(SQLException.java:353)
at
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1777)
at
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
at
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
at
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
at
org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:186)
at org.hibernate.loader.Loader.getResultSet(Loader.java:1787)
at org.hibernate.loader.Loader.scroll(Loader.java:2286)
at
org.hibernate.loader.criteria.CriteriaLoader.scroll(CriteriaLoader.java:89)
at org.hibernate.impl.SessionImpl.scroll(SessionImpl.java:1533)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:297)
at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:291)
at
com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:104)
at
com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:1)

Regards
Ankit

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Mark Kirkwood 2009-12-22 04:35:25 Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB
Previous Message Craig Ringer 2009-12-22 02:41:38 Re: URGENT HELP NEEDED. CALLING STORED PROCEDURES USING JDBC