From: | Greg Stark <stark(at)mit(dot)edu> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Andrew Dunstan <andrew(at)dunslane(dot)net>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: Odd out of memory problem. |
Date: | 2012-03-26 17:01:21 |
Message-ID: | CAM-w4HP0XGb+SWJR4gKPYDUGPf5uDFq85RORiq6tf7ttkJB+ig@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Mon, Mar 26, 2012 at 5:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Hm. This illustrates that it's not too prudent to rely on a default
> numdistinct estimate to decide that a hash aggregation is safe :-(.
> We had probably better tweak the cost estimation rules to not trust
> that. Maybe, if we have a default estimate, we should take the worst
> case estimate that the column might be unique? That could still burn
> us if the rowcount estimate was horribly wrong, but those are not nearly
> as shaky as numdistinct estimates ...
Perhaps we should have two work_mem settings -- one for the target to
aim for and one for a hard(er) limit that we should ensure the worst
case falls under?
I have a sketch for how to handle spilling hash aggregates to disk in
my head. I'm not sure if it's worth the amount of complexity it would
require but I'll poke around a bit and see if it works out well.
--
greg
From | Date | Subject | |
---|---|---|---|
Next Message | Heikki Linnakangas | 2012-03-26 17:06:37 | Re: Odd out of memory problem. |
Previous Message | Heikki Linnakangas | 2012-03-26 16:59:05 | Re: Odd out of memory problem. |