Memory allocation in spi_printtup()

From: Neil Conway <neil(dot)conway(at)gmail(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Memory allocation in spi_printtup()
Date: 2015-08-17 06:29:57
Message-ID: CAOW5sYZ-xqso3a++xG0sVVikMCt0g3Y_aOgoL5Np_9j7QcdB4A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

spi_printtup() has the following code (spi.c:1798):

if (tuptable->free == 0)
{
tuptable->free = 256;
tuptable->alloced += tuptable->free;
tuptable->vals = (HeapTuple *) repalloc(tuptable->vals,

tuptable->alloced * sizeof(HeapTuple));
}

i.e., it grows the size of the tuptable by a fixed amount when it runs
out of space. That seems odd; doubling the size of the table would be
more standard. Does anyone know if there is a rationale for this
behavior?

Attached is a one-liner to double the size of the table when space is
exhausted. Constructing a simple test case in which a large result is
materialized via SPI_execute() (e.g., by passing two large queries to
crosstab() from contrib/tablefunc), this change reduces the time
required to exceed the palloc size limit from ~300 seconds to ~93
seconds on my laptop.

Of course, using SPI_execute() rather than cursors for queries with
large result sets is not a great idea, but there is demonstrably code
that does this (e.g., contrib/tablefunc -- I'll send a patch for that
shortly).

Neil

Attachment Content-Type Size
spi-printtup-alloc-1.patch text/x-patch 490 bytes

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Heikki Linnakangas 2015-08-17 06:36:28 Re: Configure checks and optimizations/crc32 tests
Previous Message Jim Nasby 2015-08-17 04:19:13 Error message with plpgsql CONTINUE