Aggregate-function space leakage

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: pgsql-hackers(at)postgreSQL(dot)org
Cc: "Chris Spotts" <rfusca(at)gmail(dot)com>
Subject: Aggregate-function space leakage
Date: 2009-07-22 21:14:57
Message-ID: 3253.1248297297@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I looked into Chris Spotts' recent report of massive memory leakage in
8.4, in a case involving array_agg() executed in a GROUP BY query:
http://archives.postgresql.org/pgsql-general/2009-07/msg00858.php
The reason for that turns out to be that we deliberately lobotomized
array_agg that way, just last month:
http://archives.postgresql.org/pgsql-committers/2009-06/msg00259.php
in response to this problem:
http://archives.postgresql.org/pgsql-hackers/2009-06/msg01186.php

We need a better idea.

One possibility is to have nodeAgg.c reset the "aggcontext" between
groups when in group mode. I think this would be more than a one-liner
fix because it probably has pointers to transvalues that are in that
context, but it's surely doable.

Another possibility is to modify array_agg (and other aggregates that
try to use aggcontext) so that they distinguish hash- and group-mode
aggregation. If we had array_agg attach its working context to the Agg
node's per-output-tuple context instead of aggcontext when in group
mode, I think everything would work as desired. The main disadvantage
of this is modifying a coding convention that third-party code probably
is using. (But OTOH, we have already required any such code to change
for 8.4.)

I haven't tested either of the above ideas --- just brainstorming
at this point.

Comments, better ideas?

regards, tom lane

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Jeff Davis 2009-07-22 21:26:06 Re: Aggregate-function space leakage
Previous Message Robert Haas 2009-07-22 20:11:59 Re: extension facility (was: revised hstore patch)