Skip site navigation (1) Skip section navigation (2)

Re: Linux machine aggressively clearing cache

From: Dave Crooke <dcrooke(at)gmail(dot)com>
To: Joshua Berkus <josh(at)agliodbs(dot)com>
Cc: postgres performance list <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Linux machine aggressively clearing cache
Date: 2012-03-27 20:40:32
Message-ID: CALi4UpjQSQ3FeShXsLRkG8p5cyuRpJLzy7_hmLr0DRsZ_qVK+A@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-performance
This may just be a typo, but if you really did create write (dirty) block
device cache by writing the pg_dump file somewhere, then that is what it's
supposed to do ;) Linux is more aggressive about write cache and will allow
more of it to build up than e.g. HP-UX which will start to throttle
process-to-cache writes to avoid getting too far behind.

Read cache of course does not need to be flushed and can simply be dumped
when the memory is needed, and so Linux will keep more or less unlimited
amounts of read cache until it needs the memory for something else ....
here is an output from "free" on my laptop, showing ~2.5GB of read cache
that can be freed almost instantly if needed for process memory, write
cache, kernel buffers, etc. The -/+ line shows a net of what is being used
by processes.

dave:~$ free
             total       used       free     shared    buffers     cached
Mem:       8089056    7476424     612632          0     603508    2556584
-/+ buffers/cache:    4316332    3772724
Swap:     24563344    1176284   23387060

redirecting  *pg_dump >/dev/null*  will read the DB without writing
anything, but it's pretty resource intensive .... if you just want to get
the database tables into the OS read cache you can do it much more cheaply
with   *sudo tar cvf - /var/lib/postgresql/8.4/main/base | cat >/dev/null  *or
similar (GNU tar somehow detects if you connect its stdout directly to
/dev/null and then it cheats and doesn't do the reads)

In the second "free" output below, the kernel has grabbed what it can for
cache, leaving only ~64MB of actual free memory for instant use.

dave:~$ pg_dump -F c hyper9db >/dev/null
dave:~$ free
             total       used       free     shared    buffers     cached
Mem:       8089056    8024252      64804          0     287432    3797956
-/+ buffers/cache:    3938864    4150192
Swap:     24563344    1166556   23396788
dave:~$

Cheers
Dave

On Tue, Mar 27, 2012 at 3:06 PM, Joshua Berkus <josh(at)agliodbs(dot)com> wrote:

> ... but Linux rapidly cleared the cache *(flushing to disk)* down to 25GB
> within an hour.
>
> --
> Josh Berkus
> PostgreSQL Experts Inc.
> http://pgexperts.com
> San Francisco
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance
>

In response to

Responses

pgsql-performance by date

Next:From: Francois DeliegeDate: 2012-03-27 20:54:36
Subject: Re: Distinct + Limit
Previous:From: Francois DeliegeDate: 2012-03-27 20:37:51
Subject: Distinct + Limit

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group