Skip site navigation (1) Skip section navigation (2)

Re: [PERFORM] psql -A (unaligned format) eats too much

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Mark Woodward <pgsql(at)mohawksoft(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com>, Zoltan Boszormenyi <zboszor(at)dunaweb(dot)hu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Date: 2006-06-05 16:40:38
Message-ID: 44845E86.3080001@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
Mark Woodward wrote:
>> "Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
>>     
>>> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote:
>>>       
>>>> I'm reading this as just another uninformed complaint about libpq's
>>>> habit of buffering the whole query result.  It's possible that there's
>>>> a memory leak in the -A path specifically, but nothing said so far
>>>> provided any evidence for that.
>>>>         
>>> Certainly seems like it. It seems like it would be good to allow for
>>> libpq not to buffer, since there's cases where it's not needed...
>>>       
>> See past discussions.  The problem is that libpq's API says that when it
>> hands you back the completed query result, the command is complete and
>> guaranteed not to fail later.  A streaming interface could not make that
>> guarantee, so it's not a transparent substitution.
>>
>> I wouldn't have any strong objection to providing a separate API that
>> operates in a streaming fashion, but defining it is something no one's
>> bothered to do yet.  In practice, if you have to code to a variant API,
>> it's not that much more trouble to use a cursor...
>>
>>     
>
> Wouldn't the "COPY (select ...) TO STDOUT" format being discussed solve
> this for free?
>
>
>   

It won't solve it in the general case for clients that expect a result 
set. ISTM that "use a cursor" is a perfectly reasonable answer, though.

cheers

andrew


In response to

Responses

pgsql-performance by date

Next:From: Zoltan BoszormenyiDate: 2006-06-05 16:45:13
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Tom LaneDate: 2006-06-05 16:00:35
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much memory

pgsql-hackers by date

Next:From: Zoltan BoszormenyiDate: 2006-06-05 16:45:13
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Tom LaneDate: 2006-06-05 16:00:35
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much memory

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group