Skip site navigation (1) Skip section navigation (2)

Re: Vacuum and Memory Loss

From: "Jim C(dot) Nasby" <jim(at)nasby(dot)net>
To: Richard Huxton <dev(at)archonet(dot)com>
Cc: Mike <akiany(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Vacuum and Memory Loss
Date: 2006-10-23 21:36:41
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-performance
On Mon, Oct 23, 2006 at 09:45:59AM +0100, Richard Huxton wrote:
> Mike wrote:
> >Hello friends,
> >
> >I am responsible for maintaining a high volume website using postgresql
> >8.1.4. Given the amount of reads and writes, I vacuum full the server a
> >few times a week around 1, 2 AM shutting down the site for a few
> >minutes. The next day morning around 10 - 11 AM the server slows down
> >to death. It used to be that the error 'Too many clients' would be
> >recorded, until I increased the number of clients it can handle, and
> >now it simply slows down to death having lots and lots of postmaster
> >processes running:
> >
> >Tasks: 665 total,  10 running, 655 sleeping,   0 stopped,   0 zombie
> >Cpu(s): 14.9% us, 16.7% sy,  0.0% ni,  0.0% id, 68.4% wa,  0.0% hi,
> >0.0% si
> >Mem:   2074932k total,  2051572k used,    23360k free,     2736k
> >buffers
> >Swap:  2096440k total,  1844448k used,   251992k free,   102968k cached
> This seems to be saying you have 1.8GB of swap in use. I'd start by 
> checking with vmstat whether you're actively swapping. If so, you're 
> overallocating memory.

Which could easily be caused by a combination of trying to handle too
many database connections at once and setting work_mem too high.

I've often gone into client sites to find they've set the database up to
accept hundreds or thousands of connections, even though the hardware
they're running on would most likely fall over if they actually had that
many simultaneously active connections. In many cases, increasing the
number of connections the the database will hurt performance rather than
help it, because you're now asking an already overloaded server to do
even more work.
Jim Nasby                                            jim(at)nasby(dot)net
EnterpriseDB      512.569.9461 (cell)

In response to

pgsql-performance by date

Next:From: Jim C. NasbyDate: 2006-10-23 21:47:09
Subject: Re: New hardware thoughts
Previous:From: Joshua D. DrakeDate: 2006-10-23 21:33:59
Subject: Re: New hardware thoughts

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group