Skip site navigation (1) Skip section navigation (2)

Re: Out of memory

From: Alex Adriaanse <alex(at)innovacomputing(dot)com>
To: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org
Subject: Re: Out of memory
Date: 2008-03-28 21:20:01
Message-ID: 47ED6101.2060907@innovacomputing.com (view raw or flat)
Thread:
Lists: pgsql-general
Scott Marlowe wrote:
> Just because you can set max_connections to 2000 doesn't mean it's a
> good idea.  If your client needs 1000 persistent connections, then put
> a connection pooler between your app (I'm guessing php since it
> operates this way) and the database.
>
> Running 1000 connections is a LOT, and you need 1000 active
> connections, then you're likely gonna need a bigger machine than one
> with 8 cores and 16 gig of rams.  OTOH, if you are actively servicing
> less than 10% of those connections at a time, then you're wasting
> memory on the number of backends that are started up and doing
> nothing.  each one consumes some amount of memory on its own, usually
> in the 5 to 10 meg range, just to sit there and do nothing.
>
> Plus you've got issues with thundering herd type situations that can
> show up as you increase connections.
>
> Pooling is the answer here.
>   
Thanks for the suggestion.  I fully agree with you, and the client is 
already making plans to move from persistent connections to connection 
pooling (they will probably use PgBouncer).  I'm just trying to figure 
out the exact cause of this memory issue so that we can make sure it 
doesn't happen again, even after they switch to connection pooling.  
According to my rough calculations, if the server were to max out at 
2,000 connections there should still be around 2GB of memory free - 
unless all of them were running active queries which is doubtful since I 
did not see unusual changes in the load patterns.  And from what I could 
tell, the server did not tap into swap when PostgreSQL ran out of memory.

Thanks,

Alex


In response to

pgsql-general by date

Next:From: Sam MasonDate: 2008-03-28 21:21:48
Subject: Re: Escaping \n
Previous:From: Terry Lee TuckerDate: 2008-03-28 21:06:10
Subject: Escaping \n

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group