Skip site navigation (1) Skip section navigation (2)

OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB

From: Ankit Kumar <ankitk(at)xebia(dot)com>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB
Date: 2009-12-22 04:15:35
Message-ID: 4B3047E7.9020901@xebia.com (view raw or flat)
Thread:
Lists: pgsql-jdbc
Hi

I am running a Criteria.scroll() on postgresql on a DB containing 2M 
records. The memory keeps increasing and finally it generates an 
OutOfMemoryException. Please can you advice how to fix this.

*Postgresql DB version:* 8.4
*Postgresql Driver Used:* postgresql-8.4-701.jdbc4.jar

Some forums do mention that "/scroll/" is slow on Postgresql.

Is this a known issue or we need to do something specific to make scroll 
work on Postgresql?

It appears the driver class is loading all entity objects in memory and 
not releasing them as expected in a scroll operation.


Appreciate any guidance/suggestion.

*Exception StackTrace*:
java.lang.OutOfMemoryError: Java heap space
 at java.lang.Class.getDeclaredFields0(Native Method)
 at java.lang.Class.privateGetDeclaredFields(Class.java:2291)
 at java.lang.Class.getDeclaredField(Class.java:1880)
 at 
java.util.concurrent.atomic.AtomicReferenceFieldUpdater$AtomicReferenceFieldUpdaterImpl.<init>(AtomicReferenceFieldUpdater.java:181)
 at 
java.util.concurrent.atomic.AtomicReferenceFieldUpdater.newUpdater(AtomicReferenceFieldUpdater.java:65)
 at java.sql.SQLException.<clinit>(SQLException.java:353)
 at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1777)
 at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:257)
 at 
org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:479)
 at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeWithFlags(AbstractJdbc2Statement.java:367)
 at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeQuery(AbstractJdbc2Statement.java:271)
 at 
org.apache.commons.dbcp.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:93)
 at 
org.hibernate.jdbc.AbstractBatcher.getResultSet(AbstractBatcher.java:186)
 at org.hibernate.loader.Loader.getResultSet(Loader.java:1787)
 at org.hibernate.loader.Loader.scroll(Loader.java:2286)
 at 
org.hibernate.loader.criteria.CriteriaLoader.scroll(CriteriaLoader.java:89)
 at org.hibernate.impl.SessionImpl.scroll(SessionImpl.java:1533)
 at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:297)
 at org.hibernate.impl.CriteriaImpl.scroll(CriteriaImpl.java:291)
 at 
com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:104)
 at 
com.hi.openname.dao.impl.HibernateKnowledgeItemDao.loadAllCommitted(HibernateKnowledgeItemDao.java:1)

Regards
Ankit

Responses

pgsql-jdbc by date

Next:From: Mark KirkwoodDate: 2009-12-22 04:35:25
Subject: Re: OutOfMemory hibernate scroll with 2M records | Postgresql 8.4 DB
Previous:From: Craig RingerDate: 2009-12-22 02:41:38
Subject: Re: URGENT HELP NEEDED. CALLING STORED PROCEDURES USING JDBC

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group