Skip site navigation (1) Skip section navigation (2)

Re: Limit allocated memory per session

From: daveg <daveg(at)sonic(dot)net>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Euler Taveira de Oliveira <euler(at)timbira(dot)com>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: Limit allocated memory per session
Date: 2009-10-01 19:56:02
Message-ID: 20091001195601.GC32615@sonic.net (view raw or flat)
Thread:
Lists: pgsql-hackers
On Thu, Oct 01, 2009 at 11:47:43AM -0400, Tom Lane wrote:
> Euler Taveira de Oliveira <euler(at)timbira(dot)com> writes:
> > Tom Lane escreveu:
> >> daveg <daveg(at)sonic(dot)net> writes:
> >>> I'd like to propose adding a new GUC to limit the amount of memory a backend
> >>> can allocate for its own use.
> >> 
> >> Use ulimit.

> Seriously, the proposed patch introduces overhead into a place that is
> already a known hot spot, in return for not much of anything.  It will

The overhead is simply an integer addition and compare with values that are
likely already in processor caches. And this only occurs when we actually
call malloc() to get a new block, not on every palloc. So I suspect it will
not be noticable. However, I welcome any suggestion on how to test this
and actually measure the overhead if any. pg_bench? Something else?

> *not* bound backend memory use very accurately, because there is no way
> to track raw malloc() calls. And I think that 99% of users will
>  not find it useful.

The use case that motivated is a client that runs many postgresql instances
with a mostly batch/large query workload. Some of the queries are code
generated by an application and can be very complex.  A few times a month
one of these will run through 64GB of memory and oom the host. So it
seriously hurts production. Setting work_mem low enough to prevent this
results in poor query performance.

This client does not use any outside libraries that call malloc() directly.

-dg

-- 
David Gould       daveg(at)sonic(dot)net      510 536 1443    510 282 0869
If simplicity worked, the world would be overrun with insects.

In response to

pgsql-hackers by date

Next:From: Kevin GrittnerDate: 2009-10-01 20:07:51
Subject: Re: FSM search modes
Previous:From: Kevin GrittnerDate: 2009-10-01 19:43:14
Subject: Re: FSM search modes

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group