Skip site navigation (1) Skip section navigation (2)

Re: Odd out of memory problem.

From: Greg Stark <stark(at)mit(dot)edu>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Odd out of memory problem.
Date: 2012-03-26 17:01:21
Message-ID: CAM-w4HP0XGb+SWJR4gKPYDUGPf5uDFq85RORiq6tf7ttkJB+ig@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Mon, Mar 26, 2012 at 5:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Hm.  This illustrates that it's not too prudent to rely on a default
> numdistinct estimate to decide that a hash aggregation is safe :-(.
> We had probably better tweak the cost estimation rules to not trust
> that.  Maybe, if we have a default estimate, we should take the worst
> case estimate that the column might be unique?  That could still burn
> us if the rowcount estimate was horribly wrong, but those are not nearly
> as shaky as numdistinct estimates ...

Perhaps we should have two work_mem settings -- one for the target to
aim for and one for a hard(er) limit that we should ensure the worst
case falls under?

I have a sketch for how to handle spilling hash aggregates to disk in
my head. I'm not sure if it's worth the amount of complexity it would
require but I'll poke around a bit and see if it works out well.

-- 
greg

In response to

Responses

pgsql-hackers by date

Next:From: Heikki LinnakangasDate: 2012-03-26 17:06:37
Subject: Re: Odd out of memory problem.
Previous:From: Heikki LinnakangasDate: 2012-03-26 16:59:05
Subject: Re: Odd out of memory problem.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group