Skip site navigation (1) Skip section navigation (2)

Re: Linux memory zone reclaim

From: "Kevin Grittner" <Kevin(dot)Grittner(at)wicourts(dot)gov>
To: "Greg Smith" <greg(at)2ndQuadrant(dot)com>, <pgsql-performance(at)postgresql(dot)org>
Subject: Re: Linux memory zone reclaim
Date: 2012-07-30 16:43:36
Message-ID: 501673680200002500049227@gw.wicourts.gov (view raw or flat)
Thread:
Lists: pgsql-performance
Greg Smith <greg(at)2ndQuadrant(dot)com> wrote:
 
> You can tell if this is turned on like this:
> 
> echo /proc/sys/vm/zone_reclaim_mode
 
As a data point, the benchmarks I did for some of the 9.2
scalability features does not appear to have this turned on:
 
# cat /proc/sys/vm/zone_reclaim_mode
0
 
Our Linux version:
 
Linux version 2.6.32.46-0.3-default (geeko(at)buildhost) (gcc version
4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux) ) #1 SMP
2011-09-29 17:49:31 +0200
 
This has 32 cores (64 "threads" with HT) on 4 Xeon X7560 CPUs.  
 
Intel(R) Xeon(R) CPU           X7560  @ 2.27GHz
 
It has 256GB RAM on 4GB DIMMs, with each core controlling 2 DIMMs
and each core able to directly talk to every other core.  So, it
is non-uniform, but with this arrangement it is more a matter that
there is an 8GB set of memory that is "fast" for each core and the
other 97% of RAM is all accessible at the same speed.  There were
some other options for what to install on this system or how to
install it which wouldn't have kept things this tight.
 
> Install the numactl utility and you can see why it's made that
> decision:
 
We get this:
 
# numactl --hardware
available: 4 nodes (0-3)
node 0 cpus: 0 1 2 3 4 5 6 7 32 33 34 35 36 37 38 39
node 0 size: 65519 MB
node 0 free: 283 MB
node 1 cpus: 8 9 10 11 12 13 14 15 40 41 42 43 44 45 46 47
node 1 size: 65536 MB
node 1 free: 25 MB
node 2 cpus: 16 17 18 19 20 21 22 23 48 49 50 51 52 53 54 55
node 2 size: 65536 MB
node 2 free: 26 MB
node 3 cpus: 24 25 26 27 28 29 30 31 56 57 58 59 60 61 62 63
node 3 size: 65536 MB
node 3 free: 25 MB
node distances:
node   0   1   2   3 
  0:  10  11  11  11 
  1:  11  10  11  11 
  2:  11  11  10  11 
  3:  11  11  11  10 
 
When considering a hardware purchase, it might be wise to pay close
attention to how "far" a core may need to go to get to the most
"distant" RAM.
 
-Kevin

In response to

Responses

pgsql-performance by date

Next:From: Scott MarloweDate: 2012-07-30 17:09:35
Subject: Re: Linux memory zone reclaim
Previous:From: Peter GeogheganDate: 2012-07-29 15:39:21
Subject: Help me develop new commit_delay advice

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group