Skip site navigation (1) Skip section navigation (2)

Re: [PERFORM] psql -A (unaligned format) eats too much memory

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com>
Cc: Zoltan Boszormenyi <zboszor(at)dunaweb(dot)hu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much memory
Date: 2006-06-05 16:00:35
Message-ID: 20813.1149523235@sss.pgh.pa.us (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
"Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote:
>> I'm reading this as just another uninformed complaint about libpq's
>> habit of buffering the whole query result.  It's possible that there's
>> a memory leak in the -A path specifically, but nothing said so far
>> provided any evidence for that.

> Certainly seems like it. It seems like it would be good to allow for
> libpq not to buffer, since there's cases where it's not needed...

See past discussions.  The problem is that libpq's API says that when it
hands you back the completed query result, the command is complete and
guaranteed not to fail later.  A streaming interface could not make that
guarantee, so it's not a transparent substitution.

I wouldn't have any strong objection to providing a separate API that
operates in a streaming fashion, but defining it is something no one's
bothered to do yet.  In practice, if you have to code to a variant API,
it's not that much more trouble to use a cursor...

			regards, tom lane

In response to

Responses

pgsql-performance by date

Next:From: Andrew DunstanDate: 2006-06-05 16:40:38
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Jim C. NasbyDate: 2006-06-05 15:53:49
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much memory

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2006-06-05 16:40:38
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Teodor SigaevDate: 2006-06-05 15:58:02
Subject: Re: Connection Broken with Custom Dicts for TSearch2

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group