Re: [BUGS] BUG #8573: int4range memory consumption

From: Jim Nasby <decibel(at)decibel(dot)org>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Cc: g(dot)vanluffelen(at)qipc(dot)com, pgsql-hackers(at)postgreSQL(dot)org
Subject: Re: [BUGS] BUG #8573: int4range memory consumption
Date: 2013-11-01 20:21:43
Message-ID: FE8DFBB6-CE61-45FE-BCFE-34E3941D1D37@decibel.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-bugs pgsql-hackers

On Nov 1, 2013, at 2:08 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> g(dot)vanluffelen(at)qipc(dot)com writes:
>> int4range ( and any other range function) consumes much memory when used in
>> a select statement on a big table.
>
> The problem is that range_out leaks memory, as a consequence of creating a
> number of intermediate strings that it doesn't bother to free. I don't
> believe it's the only output function that leaks memory, but it does
> so with particular vim: now that we've increased the initial size of
> StringInfo buffers, it's probably wasting upwards of 2K per call.
>
> While we could doubtless hack range_out to release those strings again,
> it seems to me that that's just sticking a finger in the dike. I'm
> inclined to think that we really ought to solve this class of problems
> once and for all by fixing printtup.c to run the output functions in a
> temporary memory context,
...
> we're already using a reset-per-row approach to memory management of
> output function calls in COPY OUT, and I know for a fact that we've
> squeezed that code path as hard as we could.

+1. COPY is actually the case I was worried about… if you're dealing with large amounts of data in other clients ISTM that other things will bottleneck before the extra memory context.

In response to

Responses

Browse pgsql-bugs by date

  From Date Subject
Next Message Tomas Vondra 2013-11-02 00:16:09 Re: array_agg() on a set larger than some arbitrary(?) limit causes runaway memory usage and eventually memory exhaustion
Previous Message Tom Lane 2013-11-01 19:08:28 Re: [BUGS] BUG #8573: int4range memory consumption

Browse pgsql-hackers by date

  From Date Subject
Next Message Jim Nasby 2013-11-01 20:24:29 Re: Feature request: Optimizer improvement
Previous Message Jim Nasby 2013-11-01 20:18:02 Handle LIMIT/OFFSET before select clause (was: Feature request: optimizer improvement)