BUG #5842: Memory leak in PL/Python when taking slices of results

From: "Daniel Popowich" <danielpopowich(at)gmail(dot)com>
To: pgsql-bugs(at)postgresql(dot)org
Subject: BUG #5842: Memory leak in PL/Python when taking slices of results
Date: 2011-01-18 16:30:04
Message-ID: 201101181630.p0IGU45v047971@wwwmaster.postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers


The following bug has been logged online:

Bug reference: 5842
Logged by: Daniel Popowich
Email address: danielpopowich(at)gmail(dot)com
PostgreSQL version: 8.4.6
Operating system: x86_64-pc-linux-gnu (Ubuntu 10.04.1)
Description: Memory leak in PL/Python when taking slices of results
Details:

There is a memory leak in PL/Python when taking slices of results. This was
first discussed in pgsql-general:

http://archives.postgresql.org/pgsql-general/2011-01/msg00367.php

Thanks to Alex Hunsaker for pinpointing the problem to slices. The
following code (a modification of Alex's) demonstrates the problem well...in
a database with plpythonu installed:

-- leaks big time
CREATE or replace FUNCTION py_leak() RETURNS void
LANGUAGE plpythonu
AS $$
results = plpy.execute("""select generate_series(0, 1000000)""")
slice_creates_leak = results[:]
for r in slice_creates_leak:
pass
return
$$;

-- does not leak
CREATE or replace FUNCTION py_no_leak() RETURNS void
LANGUAGE plpythonu
AS $$
results = plpy.execute("""select generate_series(0, 1000000)""")
for noleak in results:
pass
return
$$;

I traced the bug to PLy_result_slice() in src/pl/plpython/plpython.c. That
function calls the python API function PyList_GetSlice() and erroneously
increments the reference count before returning the result to the caller.
PyList_GetSlice returns a *new* reference, not a borrowed one, so it should
just return the object as-is.

A patch is attached below.

Cheers,

Dan Popowich

----------------------------------------------------------------------

*** src/pl/plpython/plpython.c~ 2010-12-13 21:59:19.000000000 -0500
--- src/pl/plpython/plpython.c 2011-01-18 11:18:28.857831733 -0500
***************
*** 2328,2341 ****
static PyObject *
PLy_result_slice(PyObject *arg, Py_ssize_t lidx, Py_ssize_t hidx)
{
- PyObject *rv;
PLyResultObject *ob = (PLyResultObject *) arg;

! rv = PyList_GetSlice(ob->rows, lidx, hidx);
! if (rv == NULL)
! return NULL;
! Py_INCREF(rv);
! return rv;
}

static int
--- 2328,2336 ----
static PyObject *
PLy_result_slice(PyObject *arg, Py_ssize_t lidx, Py_ssize_t hidx)
{
PLyResultObject *ob = (PLyResultObject *) arg;

! return PyList_GetSlice(ob->rows, lidx, hidx);
}

static int

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2011-01-18 16:31:17 Re: pg_basebackup for streaming base backups
Previous Message Tom Lane 2011-01-18 16:15:02 Re: texteq/byteaeq: avoid detoast [REVIEW]

Browse pgsql-bugs by date

  From Date Subject
Next Message Pavel Stehule 2011-01-18 22:49:29 statement EXECUTE isn't possible call from SPI_execute_with_args function
Previous Message Anthony Manfredi 2011-01-18 16:08:42 Re: date_trunc check constraint causes errors when restoring in a db with a different time zone