Re: extremly low memory usage

From: John A Meinel <john(at)arbash-meinel(dot)com>
To: Jeremiah Jahn <jeremiah(at)cs(dot)earlham(dot)edu>
Cc: Ron <rjpeace(at)earthlink(dot)net>, postgres performance <pgsql-performance(at)postgresql(dot)org>
Subject: Re: extremly low memory usage
Date: 2005-08-21 02:26:04
Message-ID: 4307E63C.7050408@arbash-meinel.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Jeremiah Jahn wrote:
> I'm just watching gnome-system-monoitor. Which after careful
> consideration.....and looking at dstat means I'm on CRACK....GSM isn't
> showing cached memory usage....I asume that the cache memory usage is
> where data off of the disks would be cached...?
>

Well a simple "free" also tells you how much has been cached.
I believe by reading the _cach line, it looks like you have 4.6G cached.
So you are indeed using memory.

I'm still concerned why it seems to be taking 3-4ms per index lookup,
when things should already be cached in RAM.
Now, I may be wrong about whether the indexes are cached, but I sure
would expect them to be.
What is the time for a cached query on your system (with normal nested
loops)? (give the EXPLAIN ANALYZE for the *second* run, or maybe the
fourth).

I'm glad that we aren't seeing something weird with your kernel, at least.

John
=:->

>
>
> memory output from dstat is this for a few seconds:
>
> ---procs--- ------memory-usage----- ---paging-- --disk/sda----disk/sdb- ----swap--- ----total-cpu-usage----
> run blk new|_used _buff _cach _free|__in_ _out_|_read write:_read write|_used _free|usr sys idl wai hiq siq
> 0 0 0|1336M 10M 4603M 17M| 490B 833B|3823B 3503k:1607k 4285k| 160k 2048M| 4 1 89 7 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 464k| 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 48k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 132k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 36k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 12k: 0 0 | 160k 2048M| 25 0 75 0 0 0
> 1 0 0|1337M 10M 4600M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 0 75 0 0 0
> 2 0 0|1353M 10M 4585M 18M| 0 0 | 0 0 : 0 0 | 160k 2048M| 25 1 75 0 0 0
> 1 0 0|1321M 10M 4616M 19M| 0 0 | 0 0 : 0 0 | 160k 2048M| 18 8 74 0 0 0
> 1 0 0|1326M 10M 4614M 17M| 0 0 | 0 0 :4096B 0 | 160k 2048M| 16 10 74 1 0 0
> 1 0 0|1330M 10M 4609M 17M| 0 0 | 0 12k:4096B 0 | 160k 2048M| 17 9 74 0 0 0
> 0 1 0|1343M 10M 4596M 17M| 0 0 | 0 0 : 0 316M| 160k 2048M| 5 10 74 11 0 1
> 0 1 0|1339M 10M 4596M 21M| 0 0 | 0 0 : 0 0 | 160k 2048M| 0 0 74 25 0 1
> 0 2 0|1334M 10M 4596M 25M| 0 0 | 0 4096B: 0 0 | 160k 2048M| 0 0 54 44 0 1
> 1 0 0|1326M 10M 4596M 34M| 0 0 | 0 0 : 0 364k| 160k 2048M| 4 1 60 34 0 1
> 1 0 0|1290M 10M 4596M 70M| 0 0 | 0 12k: 0 0 | 160k 2048M| 24 1 75 0 0 0
> 1 0 0|1301M 10M 4596M 59M| 0 0 | 0 20k: 0 0 | 160k 2048M| 21 4 75 0 0 0
> 1 0 0|1312M 10M 4596M 48M| 0 0 | 0 0 : 0 0 | 160k 2048M| 22 4 75 0 0 0
> 1 0 0|1323M 10M 4596M 37M| 0 0 | 0 0 : 0 24k| 160k 2048M| 21 4 75 0 0 0
> 1 0 0|1334M 10M 4596M 25M| 0 0 | 0 0 : 0 56k| 160k 2048M| 21 4 75 0 0 0
>
>
>
> On Fri, 2005-08-19 at 16:07 -0500, John A Meinel wrote:
>
>>Jeremiah Jahn wrote:
>>
>>>Rebuild in progress with just ext3 on the raid array...will see if this
>>>helps the access times. If it doesn't I'll mess with the stripe size. I
>>>have REINDEXED, CLUSTERED, tablespaced and cached with 'cat table/index
>>>
>>>
>>>>/dev/null' none of this seems to have helped, or even increased my
>>>
>>>memory usage. argh! The only thing about this new system that I'm
>>>unfamiliar with is the array setup and LVM, which is why I think that's
>>>where the issue is. clustering and indexing as well as vacuum etc are
>>>things that I do and have been aware of for sometime. Perhaps slony is a
>>>factor, but I really don't see it causing problems on index read speed
>>>esp. when it's not running.
>>>
>>>thanx for your help, I really appreciate it.
>>>-jj-
>>>
>>
>>By the way, how are you measuring memory usage? Can you give the output
>>of that command, just to make sure you are reading it correctly.
>>
>>John
>>=:->
>>

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message John A Meinel 2005-08-21 02:32:04 Re: extremly low memory usage
Previous Message Ron 2005-08-20 21:23:25 Re: extremly low memory usage