Skip site navigation (1) Skip section navigation (2)

Re: "out of memory" error

From: Sean Shanny <shannyconsulting(at)earthlink(dot)net>
To: Rachel McConnell <rachel(at)enginegreen(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: "out of memory" error
Date: 2004-05-01 00:08:12
Message-ID: 4092EA6C.5030808@earthlink.net (view raw or flat)
Thread:
Lists: pgsql-general
Rachel,

Most likely your data set coming back is too large to process all at 
once.  You need to set the fetch size on the statement to control the 
number of records coming back per request for them.

ps.setFetchSize( 100 );  where ps is a prepared statement.  The result 
set will use a cursor to only bring back 100 records at a time and will 
automatically fetch the next hundred until the entire result set has 
been brought back.

Make sure that your JDBC statements don't have a semi colon at the end 
of them otherwise the setFetchSize will be ignored and the entire result 
set will be returned at once.

--sean


Rachel McConnell wrote:

> Hello,
>
> I have a Java web application using Hibernate to connect to a 
> PostgreSQL backend.  I am seeing the below stack trace during 
> processing of a set of data consisting of around 1000 objects; for a 
> 200 object set I don't see the exception.
>
> I believe the salient point is the Out Of Memory bit - marked below by 
> *****'s
> The fact that this exception occurs during a call to 
> org.postgresql.util.PSQLException.parseServerError() seems disturbing 
> too.
>
> * the JVM or servlet container might be out of memory, however the db 
> is on a different machine than the server so this doesn't seem likely
>
> * Postgres might be out of memory.  How would I check for this? 
> Subsequent requests go through just fine, as do any requests I make 
> directly using psql.
>
> * something Else is happening
>
> Any advice on how to attack this issue would be much appreciated!
>
> Rachel McConnell
>
>
> WARN  [     SystemErrorInterceptor ] - java.util.NoSuchElementException
> java.util.NoSuchElementException
>         at java.util.AbstractList$Itr.next(AbstractList.java:426)
>         at java.util.Collections$6.nextElement(Collections.java:2479)
>         at 
> com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:89) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116) 
>
>         at 
> com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182) 
>
>         at 
> com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162) 
>
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
>         at 
> com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96) 
>
>         at 
> com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57) 
>
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at 
> com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42) 
>
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at 
> com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at com.caucho.server.http.Invocation.service(Invocation.java:315)
>         at 
> com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
>         at 
> com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346) 
>
>         at 
> com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274) 
>
>         at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
>         at java.lang.Thread.run(Thread.java:534)
> WARN  [     SystemErrorInterceptor ] - 
> com.enginegreen.denali.persistence.PersistenceException: 
> net.sf.hibernate.JDBCException: could not initialize collection: 
> [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
>         at 
> com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:429) 
>
>         at 
> com.enginegreen.denali.domain.billing.BillingManagerImpl.finalizeNextBillingPeriod(BillingManagerImpl.java:248) 
>
>         at 
> com.enginegreen.denali.domain.billing.SecureBillingManager.finalizeNextBillingPeriod(SecureBillingManager.java:96) 
>
>         at 
> com.enginegreen.denali.action.billing.GenerateInvoicesAction.doExecute(GenerateInvoicesAction.java:121) 
>
>         at 
> com.enginegreen.denali.action.WorkflowActionSupport.execute(WorkflowActionSupport.java:65) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:172) 
>
>         at 
> com.enginegreen.denali.interceptor.ActionSourceInterceptor.intercept(ActionSourceInterceptor.java:50) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.enginegreen.denali.interceptor.NullInterceptor.intercept(NullInterceptor.java:25) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.AroundInterceptor.intercept(AroundInterceptor.java:37) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.enginegreen.denali.interceptor.security.AdminSecurityInterceptor.intercept(AdminSecurityInterceptor.java:31) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.enginegreen.denali.interceptor.SystemErrorInterceptor.intercept(SystemErrorInterceptor.java:47) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.interceptor.TimerInterceptor.intercept(TimerInterceptor.java:62) 
>
>         at 
> com.opensymphony.xwork.DefaultActionInvocation.invoke(DefaultActionInvocation.java:169) 
>
>         at 
> com.opensymphony.xwork.DefaultActionProxy.execute(DefaultActionProxy.java:116) 
>
>         at 
> com.opensymphony.webwork.dispatcher.ServletDispatcher.serviceAction(ServletDispatcher.java:182) 
>
>         at 
> com.opensymphony.webwork.dispatcher.ServletDispatcher.service(ServletDispatcher.java:162) 
>
>         at javax.servlet.http.HttpServlet.service(HttpServlet.java:103)
>         at 
> com.caucho.server.http.FilterChainServlet.doFilter(FilterChainServlet.java:96) 
>
>         at 
> com.enginegreen.denali.chrome.filter.ChromeFilter.doFilter(ChromeFilter.java:57) 
>
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at 
> com.enginegreen.denali.interceptor.HibernateFilter.doFilter(HibernateFilter.java:42) 
>
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at 
> com.enginegreen.denali.action.DenaliFilter.doFilter(DenaliFilter.java:28)
>         at 
> com.caucho.server.http.FilterChainFilter.doFilter(FilterChainFilter.java:88) 
>
>         at com.caucho.server.http.Invocation.service(Invocation.java:315)
>         at 
> com.caucho.server.http.CacheInvocation.service(CacheInvocation.java:135)
>         at 
> com.caucho.server.http.RunnerRequest.handleRequest(RunnerRequest.java:346) 
>
>         at 
> com.caucho.server.http.RunnerRequest.handleConnection(RunnerRequest.java:274) 
>
>         at com.caucho.server.TcpConnection.run(TcpConnection.java:139)
>         at java.lang.Thread.run(Thread.java:534)
> Caused by: net.sf.hibernate.JDBCException: could not initialize 
> collection: 
> [com.enginegreen.ledger.model.InvoiceSummary.orderSummarySet#78828]
>         at 
> net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:287) 
>
>         at 
> net.sf.hibernate.impl.SessionImpl.initializeCollection(SessionImpl.java:3226) 
>
>         at 
> net.sf.hibernate.collection.PersistentCollection.forceInitialization(PersistentCollection.java:340) 
>
>         at 
> net.sf.hibernate.impl.SessionImpl.initializeNonLazyCollections(SessionImpl.java:3089) 
>
>         at 
> net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:138) 
>
>         at net.sf.hibernate.loader.Loader.doList(Loader.java:950)
>         at net.sf.hibernate.loader.Loader.list(Loader.java:941)
>         at 
> net.sf.hibernate.hql.QueryTranslator.list(QueryTranslator.java:834)
>         at net.sf.hibernate.impl.SessionImpl.find(SessionImpl.java:1512)
>         at net.sf.hibernate.impl.QueryImpl.list(QueryImpl.java:39)
>         at 
> com.enginegreen.ledger.invoice.impl.BillingClerkImpl.renderInvoicesAsPDF(BillingClerkImpl.java:414) 
>
>         ... 42 more
>
> ******
>
> Caused by: org.postgresql.util.PSQLException: ERROR: out of memory
>
>         at 
> org.postgresql.util.PSQLException.parseServerError(PSQLException.java:139) 
>
>         at 
> org.postgresql.core.QueryExecutor.executeV3(QueryExecutor.java:154)
>         at 
> org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:101)
>         at 
> org.postgresql.core.QueryExecutor.execute(QueryExecutor.java:43)
>         at 
> org.postgresql.jdbc1.AbstractJdbc1Statement.execute(AbstractJdbc1Statement.java:515) 
>
>         at 
> org.postgresql.jdbc2.AbstractJdbc2Statement.execute(AbstractJdbc2Statement.java:50) 
>
>         at 
> org.postgresql.jdbc1.AbstractJdbc1Statement.executeQuery(AbstractJdbc1Statement.java:231) 
>
>         at 
> net.sf.hibernate.impl.BatcherImpl.getResultSet(BatcherImpl.java:87)
>         at net.sf.hibernate.loader.Loader.getResultSet(Loader.java:795)
>         at net.sf.hibernate.loader.Loader.doQuery(Loader.java:189)
>         at 
> net.sf.hibernate.loader.Loader.doQueryAndInitializeNonLazyCollections(Loader.java:133) 
>
>         at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:910)
>         at net.sf.hibernate.loader.Loader.loadCollection(Loader.java:885)
>         at 
> net.sf.hibernate.loader.OneToManyLoader.initialize(OneToManyLoader.java:80) 
>
>         at 
> net.sf.hibernate.collection.AbstractCollectionPersister.initialize(AbstractCollectionPersister.java:284) 
>
>         ... 52 more
>
>
>
>
> ---------------------------(end of broadcast)---------------------------
> TIP 1: subscribe and unsubscribe commands go to majordomo(at)postgresql(dot)org
>

In response to

pgsql-general by date

Next:From: Tom LaneDate: 2004-05-01 01:13:45
Subject: Re: One or Both Foregin Keys
Previous:From: Dennis GearonDate: 2004-04-30 23:34:07
Subject: One or Both Foregin Keys

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group