Re: Disabling memory overcommit deemed dangerous

David Geier <geidav(dot)pg(at)gmail(dot)com> writes:
> In our documentation we recommend disabling memory overcommit to prevent
> the OOM killer from kicking in, see [1]. Accordingly, we expect
> PostgreSQL to handle OOM situations gracefully. In my experience there
> are unfortunately several severe problems with that approach:

> 1. PostgreSQL contains code paths that aren't safe against failing
> memory allocations. Examples are broken cleanup code, see [2], or
> various calls to strdup() where we don't check the return value.

If you are aware of such places, please submit patches to fix them,
because they are bugs with or without overcommit. Overcommit does
*not* prevent the kernel from returning ENOMEM, so this seems like
an extremely specious argument for not telling people to disable
overcommit.

> 2. On Linux, running OOM during stack expansion triggers SIGSEGV.

Again, allowing overcommit is hardly a cure.

> 3. Other processes running on the same system are mostly not safe
> against failing memory allocations.

The overcommit recommendation is only meant for machines that are
more or less dedicated to Postgres, so I'm not sure how much this
matters. Also, we've seen comparable problems on some platforms
after running the kernel out of file descriptors. The bottom line
is that you need a reasonable amount of headroom in your system
provisioning.

> I cannot see how someone would today reliably run a PostgreSQL server
> with memory overcommit disabled, if it truly runs occasionally OOM.

We have very substantial field experience showing that leaving memory
overcommit enabled also makes the system unreliable, if it approaches
OOM conditions. I don't think removing that advice is an improvement.

regards, tom lane