Re: Linux memory zone reclaim

From: Scott Marlowe <scott(dot)marlowe(at)gmail(dot)com>
To: John Lister <john(dot)lister(at)kickstone(dot)com>
Cc: Claudio Freire <klaussfreire(at)gmail(dot)com>, pgsql-performance(at)postgresql(dot)org
Subject: Re: Linux memory zone reclaim
Date: 2012-07-24 21:57:21
Message-ID: CAOR=d=3v3xWia+ibr-QBV==DVgOix4uzCNA8M8TDTeLJ9ecUgw@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

My experience is that disabling swap and turning off zone_reclaim_mode
gets rid of any real problem for a large memory postgresql database
server. While it would be great to have a NUMA aware pgsql, I
question the solidity and reliability of the current linux kernel
implementation in a NUMA evironment, especially given the poor
behaviour of the linux kernel as regards swap behaviour.

On Tue, Jul 24, 2012 at 3:23 PM, John Lister <john(dot)lister(at)kickstone(dot)com> wrote:
> On 24/07/2012 21:12, Claudio Freire wrote:
>>
>> On Tue, Jul 24, 2012 at 3:41 PM, Claudio Freire <klaussfreire(at)gmail(dot)com>
>> wrote:
>>>
>>> On Tue, Jul 24, 2012 at 3:36 PM, John Lister <john(dot)lister(at)kickstone(dot)com>
>>> wrote:
>>>>
>>>> Do you have a suggestion about how to do that? I'm running Ubuntu 12.04
>>>> and
>>>> PG 9.1, I've modified pg_ctlcluster to cause pg_ctl to use a wrapper
>>>> script
>>>> which starts the postmaster using a numactl wrapper, but all subsequent
>>>> client processes are started with interleaving enabled as well. Any
>>>> ideas
>>>> how to make just the postmaster process start with interleaving?
>>>
>>> postmaster should call numactl right after forking:
>>> http://linux.die.net/man/2/set_mempolicy
>>
>> Something like the attached patch (untested)
>
> Cheers, I'll give it a go, I wonder if this is likely to be integrated into
> the main code? As has been mentioned here before, postgresql isn't as badly
> affected as mysql for example, but I'm wondering if the trend to larger
> memory and more cores/nodes means it should be offered as an option?
> Although saying that I've read that 10Gb of shared buffers may be enough
> even in big machines 128+Gb ram..
>
> Thoughts?
>
> John
>
>
>
>
> --
> Sent via pgsql-performance mailing list (pgsql-performance(at)postgresql(dot)org)
> To make changes to your subscription:
> http://www.postgresql.org/mailpref/pgsql-performance

--
To understand recursion, one must first understand recursion.

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Claudio Freire 2012-07-24 22:05:56 Re: Linux memory zone reclaim
Previous Message Claudio Freire 2012-07-24 21:37:44 Re: Linux memory zone reclaim