Skip site navigation (1) Skip section navigation (2)

Re: [PATCHES] ARC Memory Usage analysis

From: Greg Stark <gsstark(at)mit(dot)edu>
To: Curt Sampson <cjs(at)cynic(dot)net>
Cc: Greg Stark <gsstark(at)mit(dot)edu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PATCHES] ARC Memory Usage analysis
Date: 2004-10-26 15:30:23
Message-ID: 87r7nlts34.fsf@stark.xeocode.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-patchespgsql-performance
Curt Sampson <cjs(at)cynic(dot)net> writes:

> On Tue, 26 Oct 2004, Greg Stark wrote:
> 
> > I see mmap or O_DIRECT being the only viable long-term stable states. My
> > natural inclination was the former but after the latest thread on the subject
> > I suspect it'll be forever out of reach. That makes O_DIRECT And a Postgres
> > managed cache the only real choice. Having both caches is just a waste of
> > memory and a waste of cpu cycles.
> 
> I don't see why mmap is any more out of reach than O_DIRECT; it's not
> all that much harder to implement, and mmap (and madvise!) is more
> widely available.

Because there's no way to prevent a write-out from occurring and no way to be
notified by mmap before a write-out occurs, and Postgres wants to do its WAL
logging at that time if it hasn't already happened.

> But if using two caches is only costing us 1% in performance, there's
> not really much point....

Well firstly it depends on the work profile. It can probably get much higher
than we saw in that profile if your work load is causing more fresh buffers to
be fetched.

Secondly it also reduces the amount of cache available. If you have 256M of
ram with about 200M free, and 40Mb of ram set aside for Postgres's buffer
cache then you really only get 160Mb. It's costing you 20% of your cache, and
reducing the cache hit rate accordingly.

Thirdly the kernel doesn't know as much as Postgres about the load. Postgres
could optimize its use of cache based on whether it knows the data is being
loaded by a vacuum or sequential scan rather than an index lookup. In practice
Postgres has gone with ARC which I suppose a kernel could implement anyways,
but afaik neither linux nor BSD choose to do anything like it.

-- 
greg


In response to

Responses

pgsql-performance by date

Next:From: Tom LaneDate: 2004-10-26 16:39:44
Subject: Re: Large Database Performance suggestions
Previous:From: Joshua MarshDate: 2004-10-26 15:24:08
Subject: Re: Large Database Performance suggestions

pgsql-hackers by date

Next:From: Mike MascariDate: 2004-10-26 15:34:56
Subject: Re: making pdf of docs
Previous:From: Kris JurkaDate: 2004-10-26 15:27:39
Subject: Re: making pdf of docs

pgsql-patches by date

Next:From: Tom LaneDate: 2004-10-26 16:06:54
Subject: Re: pg_autovacuum vacuum cost variables patch
Previous:From: Tom LaneDate: 2004-10-26 14:58:47
Subject: Re: Possible make_oidjoins_check Security Issue

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group