Concern about memory management with SRFs

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Joe Conway <mail(at)joeconway(dot)com>
Cc: pgsql-hackers(at)postgreSQL(dot)org
Subject: Concern about memory management with SRFs
Date: 2002-08-28 23:08:45
Message-ID: 22310.1030576125@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

I've been looking at the sample SRFs (show_all_settings etc) and am not
happy about the way memory management is done. As the code is currently
set up, the functions essentially assume that they are executed in a
context that will never be reset until they're done returning tuples.
(This is true because tupledescs and so on are blithely constructed in
CurrentMemoryContext during the first call.)

This approach means that SRFs cannot afford to leak any memory per-call.
If they do, and the result set is large, they will run the backend out
of memory. I don't think that's acceptable.

The reason that the code fails to crash is that nodeFunctionscan.c
doesn't do a ResetExprContext(econtext) in the loop that collects rows
from the function and stashes them in the tuplestore. But I think it
must do so in the long run, and so it would be better to get this right
the first time.

I think we should document that any memory that is allocated in the
first call for use in subsequent calls must come from the memory context
saved in FuncCallContext (and let's choose a more meaningful name than
fmctx, please). This would mean adding code like

oldcontext = MemoryContextSwitchTo(funcctx->fmctx);

...

MemoryContextSwitchTo(oldcontext);

around the setup code that follows SRF_FIRSTCALL_INIT. Then it would be
safe for nodeFunctionscan.c to do a reset before each function call.

Comments?

regards, tom lane

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Robert Treat 2002-08-28 23:27:46 Re: [SQL] LIMIT 1 FOR UPDATE or FOR UPDATE LIMIT 1?
Previous Message Joe Conway 2002-08-28 22:20:22 Re: Timetable for 7.3 beta

Browse pgsql-patches by date

  From Date Subject
Next Message Matthew T. O'Connor 2002-08-29 00:15:03 Re: [HACKERS] fix for palloc() of user-supplied length
Previous Message Bruce Momjian 2002-08-28 21:33:34 Re: worried about PGPASSWORD drop