Skip site navigation (1) Skip section navigation (2)

Re: [PATCH] COPY .. COMPRESSED

From: Claudio Freire <klaussfreire(at)gmail(dot)com>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Stephen Frost <sfrost(at)snowman(dot)net>, Bruce Momjian <bruce(at)momjian(dot)us>, Peter Eisentraut <peter_e(at)gmx(dot)net>, PostgreSQL-Dev <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [PATCH] COPY .. COMPRESSED
Date: 2013-01-17 02:48:16
Message-ID: CAGTBQpb8YbVX=BHyDPXQRcQMtbn4Ai2nYCXYwNWE7wNHPNjUaw@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Wed, Jan 16, 2013 at 8:19 PM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
> On Tue, Jan 15, 2013 at 4:50 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>> I find the argument that this supports compression-over-the-wire to be
>> quite weak, because COPY is only one form of bulk data transfer, and
>> one that a lot of applications don't ever use.  If we think we need to
>> support transmission compression for ourselves, it ought to be
>> integrated at the wire protocol level, not in COPY.
>>
>> Just to not look like I'm rejecting stuff without proposing
>> alternatives, here is an idea about a backwards-compatible design for
>> doing that: we could add an option that can be set in the connection
>> request packet.  Say, "transmission_compression = gzip".
>
> But presumably this would transparently compress at one end and
> decompress at the other end, which is again a somewhat different use
> case.  To get compressed output on the client side, you have to
> decompress and recompress.  Maybe that's OK, but it's not quite the
> same thing.

Well, libpq could give some access to raw compressed streams, but,
really, even with double compression on the client, it solves the
bandwidth issue, not only for pg_dump, pg_restore, and copy, but also
for all other transfer-intensive applications. I do think it's the
best option.


In response to

pgsql-hackers by date

Next:From: Claudio FreireDate: 2013-01-17 02:56:21
Subject: Re: Parallel query execution
Previous:From: Bruce MomjianDate: 2013-01-17 02:44:53
Subject: Re: Parallel query execution

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group