Re: heavy swapping, not sure why

On Mon, Aug 29, 2011 at 3:17 PM, Merlin Moncure <mmoncure(at)gmail(dot)com> wrote:
> On Mon, Aug 29, 2011 at 4:45 PM, Lonni J Friedman <netllama(at)gmail(dot)com> wrote:
>>> using any C code in the backend? this includes 3rd party libraries
>>> which link in C, including postgis, pljava, xml2, etc.  Any features
>>> being used not included in the standard core distribution are
>>> interesting.
>>
>> Nope, nothing like that.  They're fairly generic setups, with nothing
>> added that isn't part of the core distribution.
>>
>>>
>>> How long do your database connections stay open? forever? If yes, is
>>> memory distributed semi-evenly across all postgres processes or only
>>> to particular ones? If no, do you see excessive consumption with the
>>> non user backends like the stats collector, etc?
>>
>> Nope, nothing is forever, everything is a fairly brief connection (a
>> few seconds, tops, with most under 1s). Although I do have pgbouncer
>> sitting in front of systemA to serve as a connection pooler.
>
> hm.  well iirc pgbouncer tries to dump server connections older than
> an hour or so.  this plus your other statements makes me very
> suspension the problem is in postgres itself.  since postgres process
> dies when the connection dies, long term memory accumulation is just
> not possible except in the processes that aren't serving client
> sessions (the very first thing you need to do is rule those processes
> out).  pgbouncer itself could be the issue, but i doubt it.
> obviously, a full memory profile during your problem times is a
> critical piece of evidence (a 'top' sorted by memory usage should to
> the trick nicely).

OK, I'll get that top output for everyone in a week or so once swap
usage has grown noticeably above its current level.

>
> it's possible you've unhappily tripped a leak in the o/s -- is
> everything properly updated?  running any funky hardware (like fiber
> san drivers)? anything else interesting or out of the ordinary to
> report?

No funky HW. Fairly standard 1U server with SATA disks, the OS is up to date.