Re: Memory issues

On Mon, Sep 24, 2012 at 12:45 AM, Shiran Kleiderman <shirank1(at)gmail(dot)com> wrote:
>
>
> Hi,
> I'm using and Amazon ec2 instance with the following spec and the
> application that I'm running uses a postgres DB 9.1.
> The app has 3 main cron jobs.
>
> Ubuntu 12, High-Memory Extra Large Instance
> 17.1 GB of memory
> 6.5 EC2 Compute Units (2 virtual cores with 3.25 EC2 Compute Units each)
> 420 GB of instance storage
> 64-bit platform
>
> I've changed the main default values under file postgresql.conf to:
> shared_buffers = 4GB
> work_mem = 16MB
> wal_buffers = 16MB
> checkpoint_segments = 32
> effective_cache_size = 8GB
>
> When I run the app, after an hour or two, free -m looks like below ans the
> crons can't run due to memory loss or similar (i'm new to postgres and db
> admin).
> Thanks!
>
> free -m, errors:
>
> total used free shared buffers cached
> Mem: 17079 13742 3337 0 64 11882
> -/+ buffers/cache: 1796 15283
> Swap: 511 0 511

You have 11.8G cached, that's basically free memory on demand.

> total used free shared buffers cached
> Mem: 17079 16833 245 0 42 14583
> -/+ buffers/cache: 2207 14871
> Swap: 511 0 511

Here you have 14.5G cached, again that's free memory so to speak.
I.e. when something needs it it gets allocated.

> **free above stays low even when nothing is running.
>
>
> **errors:
> DBI connect('database=---;host=localhost','postgres',...) failed: could not
> fork new process for connection: Cannot allocate memory
> could not fork new process for connection: Cannot allocate memory

This error is happening in your client process. Maybe it's 32 bit or
something and running out of local memory in its process space? Maybe
memory is so fragmented that no large blocks can get allocated or
something? Either way, your machine has plenty of memory according to
free. BTW, it's pretty common for folks new to unix to mis-read free
and not realize that cached memory + free memory is what's really
available.