Skip site navigation (1) Skip section navigation (2)

Re: Bug (and fix): leaks of TCP connections when connected

From: Oliver Jowett <oliver(at)opencloud(dot)com>
To: "Marcus Andree S(dot) Magalhaes" <marcus(dot)magalhaes(at)vlinfo(dot)com(dot)br>
Cc: sylvain(dot)laurent(at)elca(dot)ch, pgsql-jdbc(at)postgresql(dot)org
Subject: Re: Bug (and fix): leaks of TCP connections when connected
Date: 2004-06-22 03:42:53
Message-ID: 40D7AABD.10503@opencloud.com (view raw or flat)
Thread:
Lists: pgsql-jdbc
Marcus Andree S. Magalhaes wrote:
> <snip>
> 
>>In theory the discarded connections should eventually be garbage
>>collected and closed, right? So at least the leak is bounded.
> 
> I don't think so. We had a similar problem here.
> 
> After many hours of debugging, we came to the fact that either
> java garbage collector does _not_ close open connections

It's not the GC that directly closes the open connections; it just 
causes unreachable objects to be collected. It's the responsibility of 
object that holds out-of-heap resources to do whatever cleanup (OS-level 
close() calls etc) is needed when it becomes unreachable, traditionally 
via finalize() (although you can also use ReferenceQueue). I'd expect 
the socket implementation to do this.

> or do it
> after a long time (much longer than one would expect).

Typically finalization and Reference clearing happens only on full 
collections (it's not guaranteed to happen this way, though), so I'd 
expect the connections to be collected then. It can take some time for a 
full GC to happen, depending on your heap settings. We see problems in 
NIO buffer allocation when System.gc() is disabled (see below) on a 
quiescent system that can have intervals of over a day between full GCs.

If you leak sockets or some other resource rapidly enough that you run 
out of the resources at the OS level (or indirectly hit a resource 
limitation on the remote side e.g. server connection limit) before the 
owning objects are collected, you will have a problem.

NIO direct buffer allocation hits exactly this issue, and ends up 
issuing a System.gc() when it thinks it's out of direct buffer space to 
try to avoid the problem. There are some bugs in Sun's implementation 
though which make it hard to use (it forces a stop-the-world GC which is 
bad if you are using the CMS collector, and there is a race between the 
allocating thread returning from System.gc() and buffer references being 
cleared in the reference-handler thread). Assuming those problems can be 
fixed, there's an argument for doing the same thing in the socket 
allocation code (and anywhere else that allocates out-of-heap resources) 
too. It becomes a question of how far to take it: if the server refuses 
a connection with "too many connections", should you GC and try again if 
that collected any connections to the server in question?

See http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=5025281

(apologies for rambling :)

-O

In response to

pgsql-jdbc by date

Next:From: Laurent SylvainDate: 2004-06-22 07:15:57
Subject: Re: Bug (and fix): leaks of TCP connections when connected
Previous:From: Marcus Andree S. MagalhaesDate: 2004-06-22 02:57:06
Subject: Re: Bug (and fix): leaks of TCP connections when connected

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group