High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray

From: Atul Deopujari <atuldeopujari(at)gmail(dot)com>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray
Date: 2019-03-04 13:09:11
Message-ID: CAGkoPheL6vFLtM=71Vn+8sdrRj9LaRyFuc+E_U=5R5z=FdNNEQ@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-jdbc

Hello
I am trying to figure out reasons for high memory utilization by
PgConnection object. My application occasionally runs out of memory and the
heap dump shows PgConnection as one of the largest consumers of memory.

The heap dump shows PgConnection having a retained size of 640 MB.
We use a connection pool of 8 connections. Each connection object is
retaining about 80 MB. If I drill down into the PgConnection object, I see
that QueryExecutorImpl taking almost all memory within which it is the
pgStream.encoding.decoderArray taking all the memory.

Not all connection objects are 80 MB all the time. Sometimes the heap dump
also shows that some connection objects are as low as 3 MB in size.

I would like to understand under what circumstances can
pgStream.encoding.decoderArray retain 80MB. I have checked places in the
code and have ensured that we are doing resultSet.close, stmt.close and
conn.close wherever we are using them. We do have big objects in the
database. I have tried to reproduce this problem using a simple program
that queries tuples that are large text, some are even more than 500MB. I
haven't been able to see such high memory usage of QueryExecutorImpl in my
local setup.

We use the following JDBC driver and the server

- JDBC driver build number: 42.2.2
- Server version: 10.4 (Ubuntu 10.4-2.pgdg14.04+1)

Will appreciate if someone can throw light on the reasons for excessive
object retention in QueryExecutorImpl.pgStream.encoding.decoderArray which
can help me dig this problem further.

Thanks,
Atul

Responses

Browse pgsql-jdbc by date

  From Date Subject
Next Message Dave Cramer 2019-03-05 11:01:05 Re: High memory retention by QueryExecutorImpl.pgStream.encoding.decoderArray
Previous Message Švorc Martin 2019-03-01 10:16:39 RE: How to send queries to master and failover to slave ?