Skip site navigation (1) Skip section navigation (2)

BUG #5842: Memory leak in PL/Python when taking slices of results

From: "Daniel Popowich" <danielpopowich(at)gmail(dot)com>
To: pgsql-bugs(at)postgresql(dot)org
Subject: BUG #5842: Memory leak in PL/Python when taking slices of results
Date: 2011-01-18 16:30:04
Message-ID: 201101181630.p0IGU45v047971@wwwmaster.postgresql.org (view raw or flat)
Thread:
Lists: pgsql-bugspgsql-hackers
The following bug has been logged online:

Bug reference:      5842
Logged by:          Daniel Popowich
Email address:      danielpopowich(at)gmail(dot)com
PostgreSQL version: 8.4.6
Operating system:   x86_64-pc-linux-gnu (Ubuntu 10.04.1)
Description:        Memory leak in PL/Python when taking slices of results
Details: 

There is a memory leak in PL/Python when taking slices of results.  This was
first discussed in pgsql-general:

  http://archives.postgresql.org/pgsql-general/2011-01/msg00367.php

Thanks to Alex Hunsaker for pinpointing the problem to slices.  The
following code (a modification of Alex's) demonstrates the problem well...in
a database with plpythonu installed:

    -- leaks big time
    CREATE  or replace FUNCTION py_leak() RETURNS void
       LANGUAGE plpythonu
       AS $$
    results = plpy.execute("""select generate_series(0, 1000000)""")
    slice_creates_leak = results[:]
    for r in slice_creates_leak:
        pass
    return
    $$;

    -- does not leak
    CREATE  or replace FUNCTION py_no_leak() RETURNS void
       LANGUAGE plpythonu
       AS $$
    results = plpy.execute("""select generate_series(0, 1000000)""")
    for noleak in results:
        pass
    return
    $$;


I traced the bug to PLy_result_slice() in src/pl/plpython/plpython.c.  That
function calls the python API function PyList_GetSlice() and erroneously
increments the reference count before returning the result to the caller. 
PyList_GetSlice returns a *new* reference, not a borrowed one, so it should
just return the object as-is.

A patch is attached below.

Cheers,

Dan Popowich


----------------------------------------------------------------------


*** src/pl/plpython/plpython.c~	2010-12-13 21:59:19.000000000 -0500
--- src/pl/plpython/plpython.c	2011-01-18 11:18:28.857831733 -0500
***************
*** 2328,2341 ****
  static PyObject *
  PLy_result_slice(PyObject *arg, Py_ssize_t lidx, Py_ssize_t hidx)
  {
- 	PyObject   *rv;
  	PLyResultObject *ob = (PLyResultObject *) arg;
  
! 	rv = PyList_GetSlice(ob->rows, lidx, hidx);
! 	if (rv == NULL)
! 		return NULL;
! 	Py_INCREF(rv);
! 	return rv;
  }
  
  static int
--- 2328,2336 ----
  static PyObject *
  PLy_result_slice(PyObject *arg, Py_ssize_t lidx, Py_ssize_t hidx)
  {
  	PLyResultObject *ob = (PLyResultObject *) arg;
  
! 	return PyList_GetSlice(ob->rows, lidx, hidx);
  }
  
  static int

Responses

pgsql-hackers by date

Next:From: Tom LaneDate: 2011-01-18 16:31:17
Subject: Re: pg_basebackup for streaming base backups
Previous:From: Tom LaneDate: 2011-01-18 16:15:02
Subject: Re: texteq/byteaeq: avoid detoast [REVIEW]

pgsql-bugs by date

Next:From: Pavel StehuleDate: 2011-01-18 22:49:29
Subject: statement EXECUTE isn't possible call from SPI_execute_with_args function
Previous:From: Anthony ManfrediDate: 2011-01-18 16:08:42
Subject: Re: date_trunc check constraint causes errors when restoring in a db with a different time zone

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group