Skip site navigation (1) Skip section navigation (2)

Hung JDBC connections

From: Mike Charnoky <noky(at)nextbus(dot)com>
To: pgsql-jdbc(at)postgresql(dot)org
Subject: Hung JDBC connections
Date: 2012-06-08 14:20:47
Message-ID: 4FD20A3F.70602@nextbus.com (view raw or flat)
Thread:
Lists: pgsql-jdbc
Hi,

Yesterday we experienced a problem whereby hundreds of JDBC client 
connections remained in a hung state, despite the fact that the 
PostgreSQL server was actually rebooted. This is disconcerting because 
our client applications have a db failover facility, but that feature 
won't work unless existing JDBC connections (most of which are pooled) 
fail in the first place. We had to instead manually restart all client 
applications to eliminate all the hung connections.

Here are the details:
* server: PostgreSQL v8.3 on CentOS 5.4
* clients: JDBC v9.0-802.jdbc4, Java 1.6.0_30, on various CentOS 5.x 
releases

The strange thing is that on the client side, according to "netstat" the 
TCP connections between the JDBC clients and PostgreSQL were still in 
the ESTABLISHED state (even though the server was rebooted). A thread 
dump of the a typical hung client (one performing DB updates) looked 
like this, it was stuck in a socket read. I was further surprised that 
some test clients (which I did not restart yesterday) are STILL in this 
hung state this morning, 18 hours after the incident!

"DBThrottle-" prio=10 tid=0xad1ff400 nid=0x24b8 runnable [0xad2ad000]
    java.lang.Thread.State: RUNNABLE
         at java.net.SocketInputStream.socketRead0(Native Method)
         at java.net.SocketInputStream.read(SocketInputStream.java:129)
         at 
org.postgresql.core.VisibleBufferedInputStream.readMore(VisibleBufferedInputStream.java:145)
         at 
org.postgresql.core.VisibleBufferedInputStream.ensureBytes(VisibleBufferedInputStream.java:114)
         at 
org.postgresql.core.VisibleBufferedInputStream.read(VisibleBufferedInputStream.java:73)
         at org.postgresql.core.PGStream.ReceiveChar(PGStream.java:274)
         at 
org.postgresql.core.v3.QueryExecutorImpl.processResults(QueryExecutorImpl.java:1660)
         at 
org.postgresql.core.v3.QueryExecutorImpl.execute(QueryExecutorImpl.java:407)
         - locked <0xb243db20> (a org.postgresql.core.v3.QueryExecutorImpl)
         at 
org.postgresql.jdbc2.AbstractJdbc2Statement.executeBatch(AbstractJdbc2Statement.java:2725)
         at xxx.xxxxxxx.util.DBThrottle.sqlLoop(DBThrottle.java:401)
         at xxx.xxxxxxx.util.DBThrottle.access$000(DBThrottle.java:59)
         at xxx.xxxxxxx.util.DBThrottle$1.run(DBThrottle.java:199)

Is there a way this hang can be addressed by the JDBC driver? Again, 
this is very disconcerting that the clients did not detect the server 
failure. Does the connection use any sort of timeout mechanism? Thank 
you for your help.


Mike


Responses

pgsql-jdbc by date

Next:From: Maciek SakrejdaDate: 2012-06-08 16:52:48
Subject: Re: get/setReadOnly broken if default_transaction_read_only on
Previous:From: Tom LaneDate: 2012-06-08 13:47:19
Subject: Re: get/setReadOnly broken if default_transaction_read_only on

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group