Re: improve overcommit docs

At the time I wrote the original 2.6 was not out even in prerelease,
which is why I was deliberately somewhat vague about it. It is still in
prerelease, and it will in fact work slightly differently from what was
in some 2.4 kernels - there are 2 settings that govern this instead of
1. Here is the 2.6 description straight from
linux-2.6.0-test9/Documentation/vm/overcommit-accounting:

-------------------------------------------
The Linux kernel supports three overcommit handling modes

0 - Heuristic overcommit handling. Obvious overcommits of
address space are refused. Used for a typical system. It
ensures a seriously wild allocation fails while allowing
overcommit to reduce swap usage. root is allowed to
allocate slighly more memory in this mode. This is the
default.

1 - No overcommit handling. Appropriate for some scientific
applications.

2 - (NEW) strict overcommit. The total address space commit
for the system is not permitted to exceed swap + a
configurable percentage (default is 50) of physical RAM.
Depending on the percentage you use, in most situations
this means a process will not be killed while accessing
pages but will receive errors on memory allocation as
appropriate.

The overcommit policy is set via the sysctl `vm.overcommit_memory'.

The overcommit percentage is set via `vm.overcommit_ratio'.
-----------------------------------------------------

Also note that this is wrong for 2.4:

run the machine out of memory. If your kernel supports the strict
! paranoid modes of overcommit handling, you can also relieve this

There are 2 modes: strict (allow commit up to sizeof(swap plus 1/2 RAM) ) and paranoid (allow commit up to sizeof(swap) ).

Wordsmith it however you like

cheers

andrew

Neil Conway wrote:

>This patch makes some improvements to the section of the documentation
>that describes the Linux 2.4 memory overcommit behavior.
>
>I removed the almost content-free assertion that "You will need enough
>swap space to cover your memory needs." If this is intended to
>communicate anything meaningful, can someone rephrase it, please?
>
>This patch also includes a fix for a typo noticed by Robert Treat.
>
>Is this suitable for 7.4 (either the whole patch, or just the typo
>fix)?
>
>-Neil
>
>
>------------------------------------------------------------------------
>
>Index: doc/src/sgml/runtime.sgml
>===================================================================
>RCS file: /var/lib/cvs/pgsql-server/doc/src/sgml/runtime.sgml,v
>retrieving revision 1.218
>diff -c -r1.218 runtime.sgml
>*** doc/src/sgml/runtime.sgml 14 Nov 2003 15:43:22 -0000 1.218
>--- doc/src/sgml/runtime.sgml 16 Nov 2003 02:07:42 -0000
>***************
>*** 1294,1300 ****
> <para>
> Unfortunately, there is no well-defined method for determining
> ideal values for the family of <quote>cost</quote> variables that
>! below. You are encouraged to experiment and share
> your findings.
> </para>
> </note>
>--- 1294,1300 ----
> <para>
> Unfortunately, there is no well-defined method for determining
> ideal values for the family of <quote>cost</quote> variables that
>! appear below. You are encouraged to experiment and share
> your findings.
> </para>
> </note>
>***************
>*** 3267,3301 ****
> <title>Linux Memory Overcommit</title>
>
> <para>
>! Linux kernels of version 2.4.* have a poor default memory
>! overcommit behavior, which can result in the PostgreSQL server
>! (<filename>postmaster</filename> process) being killed by the
>! kernel if the memory demands of another process cause the system
>! to run out of memory.
> </para>
>
> <para>
>! If this happens, you will see a kernel message looking like this
>! (consult your system documentation and configuration on where to
>! look for such a message):
> <programlisting>
> Out of Memory: Killed process 12345 (postmaster).
> </programlisting>
>! And, of course, you will find that your database server has
>! disappeared.
> </para>
>
> <para>
> To avoid this situation, run <productname>PostgreSQL</productname>
> on a machine where you can be sure that other processes will not
> run the machine out of memory. If your kernel supports the strict
>! and/or paranoid modes of overcommit handling, you can also relieve
>! this problem by altering the system's default behaviour. This can
>! be determined by examining the function
>! <function>vm_enough_memory</> in the file <filename>mm/mmap.c</>
>! in the kernel source. If this file reveals that the strict and/or
>! paranoid modes are supported by your kernel, turn one of these
>! modes on by using
> <programlisting>
> sysctl -w vm.overcommit_memory=2
> </programlisting>
>--- 3267,3302 ----
> <title>Linux Memory Overcommit</title>
>
> <para>
>! In Linux 2.4, the default virtual memory configuration is not
>! optimal for <productname>PostgreSQL</productname>. Because of the
>! way that the kernel implements memory overcommit, the kernel may
>! terminate the <productname>PostgreSQL</productname> server (the
>! <filename>postmaster</filename> process) if the memory demands of
>! another process cause the system to run out of virtual memory.
> </para>
>
> <para>
>! If this happens, you will see a kernel message that looks like
>! this (consult your system documentation and configuration on where
>! to look for such a message):
> <programlisting>
> Out of Memory: Killed process 12345 (postmaster).
> </programlisting>
>! This indicates that the <filename>postmaster</filename> process
>! has been terminated due to memory pressure;
>! <productname>PostgreSQL</productname> will need to be restarted.
> </para>
>
> <para>
> To avoid this situation, run <productname>PostgreSQL</productname>
> on a machine where you can be sure that other processes will not
> run the machine out of memory. If your kernel supports the strict
>! paranoid modes of overcommit handling, you can also relieve this
>! problem by altering the system's default behaviour. This can be
>! determined by examining the function <function>vm_enough_memory</>
>! in the file <filename>mm/mmap.c</> in the kernel source. If this
>! file reveals that the strict or paranoid modes are supported by
>! your kernel, turn one of these modes on by using
> <programlisting>
> sysctl -w vm.overcommit_memory=2
> </programlisting>
>***************
>*** 3304,3315 ****
> sysctl -w vm.overcommit_memory=3
> </programlisting>
> for paranoid mode, or placing an equivalent entry in
>! <filename>/etc/sysctl.conf</>.
> </para>
>
> <caution>
> <para>
>! Using these settings in a kernel which does not support these
> modes will almost certainly increase the danger of the kernel
> killing the database server, rather than reducing it. If in any
> doubt, consult a kernel expert or your kernel vendor.
>--- 3305,3320 ----
> sysctl -w vm.overcommit_memory=3
> </programlisting>
> for paranoid mode, or placing an equivalent entry in
>! <filename>/etc/sysctl.conf</>. For more information on memory
>! overcommit handling and the meaning of these kernel settings,
>! consult the file
>! <filename>Documentation/vm/overcommit-accounting</filename> in a
>! recent version of the source code for the Linux kernel.
> </para>
>
> <caution>
> <para>
>! Using these settings with a kernel which does not support these
> modes will almost certainly increase the danger of the kernel
> killing the database server, rather than reducing it. If in any
> doubt, consult a kernel expert or your kernel vendor.
>***************
>*** 3317,3331 ****
> </caution>
>
> <para>
>! These modes are expected to be supported in all 2.6 and later
>! kernels. Some vendor 2.4 kernels may also support these modes. It
>! is, however, known that some vendor documents suggest that they
>! support them while examination of the kernel source reveals that
>! they do not.
>! </para>
>!
>! <para>
>! Note, you will need enough swap space to cover all your memory needs.
> </para>
> </sect2>
> </sect1>
>--- 3322,3333 ----
> </caution>
>
> <para>
>! The method described above for changing the kernel's overcommit
>! behavior should work with Linux kernel 2.6 and later. In addition,
>! some versions of the 2.4 kernel distributed by some vendors may
>! also support this functionality. However, it is known that some
>! vendor documents suggest that they support them while examination
>! of the kernel source reveals that they do not.
> </para>
> </sect2>
> </sect1>
>
>
>------------------------------------------------------------------------
>
>
>---------------------------(end of broadcast)---------------------------
>TIP 2: you can get off all lists at once with the unregister command
> (send "unregister YourEmailAddressHere" to majordomo(at)postgresql(dot)org)
>
>