Skip site navigation (1) Skip section navigation (2)

Re: 2nd Level Buffer Cache

From: Radosław Smogura <rsmogura(at)softperience(dot)eu>
To: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
Cc: "PG Hackers" <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)commandprompt(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jim Nasby <jim(at)nasby(dot)net>
Subject: Re: 2nd Level Buffer Cache
Date: 2011-03-18 23:35:00
Message-ID: 201103190035.01030.rsmogura@softperience.eu (view raw or flat)
Thread:
Lists: pgsql-hackers
"Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov> Thursday 17 March 2011 22:02:18
> Rados*aw Smogura<rsmogura(at)softperience(dot)eu> wrote:
> > I have implemented initial concept of 2nd level cache. Idea is to
> > keep some segments of shared memory for special buffers (e.g.
> > indices) to prevent overwrite those by other operations. I added
> > those functionality to nbtree index scan.
> > 
> > I tested this with doing index scan, seq read, drop system
> > buffers, do index scan and in few places I saw performance
> > improvements, but actually, I'm not sure if this was just "random"
> > or intended improvement.
> 
> I've often wondered about this.  In a database I developed back in
> the '80s it was clearly a win to have a special cache for index
> entries and other special pages closer to the database than the
> general cache.  A couple things have changed since the '80s (I mean,
> besides my waistline and hair color), and PostgreSQL has many
> differences from that other database, so I haven't been sure it
> would help as much, but I have wondered.
> 
> I can't really look at this for a couple weeks, but I'm definitely
> interested.  I suggest that you add this to the next CommitFest as a
> WIP patch, under the Performance category.
> 
> https://commitfest.postgresql.org/action/commitfest_view/open
> 
> > There is few places to optimize code as well, and patch need many
> > work, but may you see it and give opinions?
> 
> For something like this it makes perfect sense to show "proof of
> concept" before trying to cover everything.
> 
> -Kevin

Here I attach latest version of patch with few performance improvements (code 
is still dirty) and some reports from test, as well my simple tests.

Actually there is small improvement without dropping system caches, and bigger 
with dropping. I have small performance decrease (if we can talk about 
measuring basing on this tests) to original PG version when dealing with same 
configuration, but increase is with 2nd level buffers... or maybe I badly 
compared reports.

In tests I tried to choose typical, simple queries. 

Regards,
Radek

Attachment: reports_20110318.tar.bz2
Description: application/x-bzip-compressed-tar (2.5 KB)
Attachment: test-scritps_20110319_0026.tar.bz2
Description: application/x-bzip-compressed-tar (2.4 KB)
Attachment: 2nd_lvl_cache_20110318.diff.bz2
Description: application/x-bzip (9.9 KB)

In response to

pgsql-hackers by date

Next:From: Josh BerkusDate: 2011-03-18 23:55:29
Subject: Re: 2nd Level Buffer Cache
Previous:From: Bruce MomjianDate: 2011-03-18 23:23:16
Subject: pg_last_xact_replay_timestamp meaning

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group