Skip site navigation (1) Skip section navigation (2)

Re: [PERFORM] psql -A (unaligned format) eats too much

From: "Mark Woodward" <pgsql(at)mohawksoft(dot)com>
To: "Andrew Dunstan" <andrew(at)dunslane(dot)net>
Cc: "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>, "Jim C(dot) Nasby" <jnasby(at)pervasive(dot)com>, "Zoltan Boszormenyi" <zboszor(at)dunaweb(dot)hu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Date: 2006-06-05 17:01:45
Message-ID: 18067.24.91.171.78.1149526905.squirrel@mail.mohawksoft.com (view raw or flat)
Thread:
Lists: pgsql-hackerspgsql-performance
> Mark Woodward wrote:
>>> "Jim C. Nasby" <jnasby(at)pervasive(dot)com> writes:
>>>
>>>> On Mon, Jun 05, 2006 at 11:27:30AM -0400, Tom Lane wrote:
>>>>
>>>>> I'm reading this as just another uninformed complaint about libpq's
>>>>> habit of buffering the whole query result.  It's possible that
>>>>> there's
>>>>> a memory leak in the -A path specifically, but nothing said so far
>>>>> provided any evidence for that.
>>>>>
>>>> Certainly seems like it. It seems like it would be good to allow for
>>>> libpq not to buffer, since there's cases where it's not needed...
>>>>
>>> See past discussions.  The problem is that libpq's API says that when
>>> it
>>> hands you back the completed query result, the command is complete and
>>> guaranteed not to fail later.  A streaming interface could not make
>>> that
>>> guarantee, so it's not a transparent substitution.
>>>
>>> I wouldn't have any strong objection to providing a separate API that
>>> operates in a streaming fashion, but defining it is something no one's
>>> bothered to do yet.  In practice, if you have to code to a variant API,
>>> it's not that much more trouble to use a cursor...
>>>
>>>
>>
>> Wouldn't the "COPY (select ...) TO STDOUT" format being discussed solve
>> this for free?
>>
>>
>>
>
> It won't solve it in the general case for clients that expect a result
> set. ISTM that "use a cursor" is a perfectly reasonable answer, though.

I'm not sure I agree -- surprise!

psql is often used as a command line tool and using a cursor is not
acceptable.

Granted, with an unaligned output, perhaps psql should not buffer the
WHOLE result at once, but without rewriting that behavior, a COPY from
query may be close enough.

In response to

Responses

pgsql-performance by date

Next:From: Andrew DunstanDate: 2006-06-05 17:03:24
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Mark WoodwardDate: 2006-06-05 16:48:12
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much

pgsql-hackers by date

Next:From: Andrew DunstanDate: 2006-06-05 17:03:24
Subject: Re: [PERFORM] psql -A (unaligned format) eats too much
Previous:From: Zdenek KotalaDate: 2006-06-05 16:55:50
Subject: Re: Allow commenting of variables in postgresql.conf to -

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group