Skip site navigation (1) Skip section navigation (2)

Re: Patch for memory leaks in index scan

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Dmitry Tkach <dmitry(at)openratings(dot)com>
Cc: pgsql-bugs(at)postgresql(dot)org, pgsql-patches(at)postgresql(dot)org
Subject: Re: Patch for memory leaks in index scan
Date: 2002-06-23 21:29:34
Message-ID: 200206232129.g5NLTYs17982@candle.pha.pa.us (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-patches
Patch applied.  Thanks.

---------------------------------------------------------------------------


Dmitry Tkach wrote:
> Ok, this sit around for a while, and, because there was no responses, I assume, that nothing jumps out
> at you as being terribly with my logic...
> Here is the patch (see the original problem description in the bottom)... It seems to be working (at least that query,
> that used to be running out of memory is now able to finish).
> 
> *** nodeIndexscan.c.orig        Fri Apr 19 10:29:57 2002
> --- nodeIndexscan.c     Fri Apr 19 10:30:00 2002
> ***************
> *** 505,510 ****
> --- 505,514 ----
>           */
>          ExecClearTuple(scanstate->cstate.cs_ResultTupleSlot);
>          ExecClearTuple(scanstate->css_ScanTupleSlot);
> +       pfree(scanstate);
> +       pfree(indexstate->iss_RelationDescs);
> +       pfree(indexstate->iss_ScanDescs);
> +       pfree(indexstate);
>    }
> 
>    /* ----------------------------------------------------------------
> 
> I hope, it helps....
> 
> Dima
> 
> 
> -------- Original Message --------
> 
> It seems that ExecInit/EndIndexScan is leaking some memory...
> 
> For example, if I run a query, that uses an index scan, and call MemoryContextStats (CurrentMemoryContext) before
> ExecutorStart() and after ExecutorEnd() in ProcessQuery(), I am consistently seeing that the 'after' call shows
> 256 bytes more used, then 'before'...
> 
> In many common cases, this is not a problem, because pg_exec_query_string () will call
> MemoryContextResetAndDeleteChildren(), that will clean everything up eventually...
> 
> But it still seems to cause problems, when the index scan is not directly invoked from pg_exec_query_string ().
> For example:
> 
> create table a
> (
>      id int primary key,
>      data char(100)
> );
> 
> create table b
> (
>      id int references a,
>      more_data char(100)
> );
> 
> create function merge_data (int,data) returns char(200) as 'select data || $2 from a where id = $1;' language 'sql';
> 
> Now, if b is large enough, then something like:
> 
> select merge_data (id,more_data) from b;
> 
> Will eventually run out of memory - it will loose 256 bytes after each row, or about a GIG after 4 million rows...
> 
> The problem seems to be in ExecEndIndexScan - it does not release scanstate, indexstate, indexstate->iss_RelationDescs
> and indexstate -> iss_ScanDescs...
> 
> I am not familiar enough with the core code, to just jump in and start fixing it, but I can make the patch,
> test it and send it to you, if you guys let me know if what I am saying makes sense to you...
> Or am I missing something here?
> 
> Thanks!
> 
> Dima
> 
> 
> 
> ---------------------------(end of broadcast)---------------------------
> TIP 3: if posting/reading through Usenet, please send an appropriate
> subscribe-nomail command to majordomo(at)postgresql(dot)org so that your
> message can get through to the mailing list cleanly
> 

-- 
  Bruce Momjian                        |  http://candle.pha.pa.us
  pgman(at)candle(dot)pha(dot)pa(dot)us               |  (610) 853-3000
  +  If your life is a hard drive,     |  830 Blythe Avenue
  +  Christ can be your backup.        |  Drexel Hill, Pennsylvania 19026



In response to

pgsql-bugs by date

Next:From: Peter EisentrautDate: 2002-06-23 21:49:55
Subject: Re: [PATCHES] pg_dumpall should permit quiet operation
Previous:From: Thomas CastonzoDate: 2002-06-22 03:28:14
Subject: MAC OSX Question

pgsql-patches by date

Next:From: Bruce MomjianDate: 2002-06-23 21:33:43
Subject: Re: several minor cleanups
Previous:From: Bruce MomjianDate: 2002-06-23 21:26:26
Subject: Re: rtree memory leak patch

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group