Re: Built-in support for a memory consumption ulimit?

On Sat, Jun 14, 2014 at 10:37:36AM -0400, Tom Lane wrote:
> After giving somebody advice, for the Nth time, to install a
> memory-consumption ulimit instead of leaving his database to the tender
> mercies of the Linux OOM killer, it occurred to me to wonder why we don't
> provide a built-in feature for that, comparable to the "ulimit -c max"
> option that already exists in pg_ctl.

In principle, I would like to have such a feature. The only limit I've found
reliable on the occasions I wanted this was RLIMIT_AS; RLIMIT_DATA has been
ineffective on Linux+GNU libc. RLIMIT_AS has its own problems, of course:
address space usage is only tenuously connected to the definition of "memory
consumption" folks actually want. Worse, one can often construct a query to
crash an RLIMIT_AS-affected backend. Make the query use heap space until just
shy of the limit, then burn stack space until RLIMIT_AS halts stack growth.

I would welcome a feature for configuring each RLIMIT_* or some selection
thereof. Then it's up to the administrator to know the (possibly
platform-specific) benefits and risks of each limit. I don't think a high
level "limit memory consumption" feature is within reach, though.

> A reasonably low-overhead way
> to do that would be to define it as something a backend process sets
> once at startup, if told to by a GUC. The GUC could possibly be
> PGC_BACKEND level though I'm not sure if we want unprivileged users
> messing with it.

Letting unprivileged users raise the limit is somewhat like allowing them to
raise work_mem. On the other hand, while one can craft queries to consume
arbitrary multiples of work_mem, the combination of RLIMIT_AS and CONNECTION
LIMIT should be effective to cap a user's memory consumption. Overall,
PGC_SUSET sounds good to me, for the reasons Craig gave.

Thanks,
nm

--
Noah Misch
EnterpriseDB http://www.enterprisedb.com