Skip site navigation (1) Skip section navigation (2)

Re: PostgreSQL and HugePage

From: Mark Wong <markwkm(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: daveg <daveg(at)sonic(dot)net>, Greg Stark <gsstark(at)mit(dot)edu>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Mark Kirkwood <mark(dot)kirkwood(at)catalyst(dot)net(dot)nz>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: PostgreSQL and HugePage
Date: 2010-10-21 15:11:17
Message-ID: AANLkTinZOBZGUXGENrp-MvgFnRTc5BSYUFS-Jan8VE+G@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Wed, Oct 20, 2010 at 1:13 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Wed, Oct 20, 2010 at 3:47 PM, daveg <daveg(at)sonic(dot)net> wrote:
>> On Wed, Oct 20, 2010 at 12:28:25PM -0700, Greg Stark wrote:
>>> On Wed, Oct 20, 2010 at 12:17 PM, Greg Stark <gsstark(at)mit(dot)edu> wrote:
>>> > I don't think it's a big cost once all the processes
>>> > have been forked if you're reusing them beyond perhaps slightly more
>>> > efficient cache usage.
>>>
>>> Hm, this site claims to get a 13% win just from the reduced tlb misses
>>> using a preload hack with Pg 8.2. That would be pretty substantial.
>>>
>>> http://oss.linbit.com/hugetlb/
>>
>> That was my motivation in trying a patch. TLB misses can be a substantial
>> overhead. I'm not current on the state of play, but working at Sun's
>> benchmark lab on a DB TPC-B benchmark something for the first generation
>> of MP systems, something like 30% of all bus traffic was TLB misses. The
>> next iteration of the hardward had a much larger TLB.
>>
>> I have a client with 512GB memory systems, currently with 128GB configured
>> as postgresql buffer cache. Which is 32M TLB entires trying to fit in the
>> few dozed cpu TLB slots. I suspect there may be some contention.
>>
>> I'll benchmark of course.
>
> Do you mean 128GB shared buffers, or shared buffers + OS cache?  I
> think that the general wisdom is that performance tails off beyond
> 8-10GB of shared buffers anyway, so a performance improvement on 128GB
> shared buffers might not mean much unless you can also show that 128GB
> shared buffers actually performs better than some smaller amount.

I'm sure someone will correct me if I'm wrong, but when I looked at
this a couple years ago I believe a side effect of using hugetlbs is
that these segments are never swapped out.

I made a weak attempt to patch postgres to use hugetlbs when
allocating shared memory.  If I can find that patch I'll send it out..

Mark

In response to

pgsql-hackers by date

Next:From: Dimitri FontaineDate: 2010-10-21 15:12:02
Subject: Re: Extensions, this time with a patch
Previous:From: David E. WheelerDate: 2010-10-21 15:00:51
Subject: Re: Extensions, this time with a patch

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group