Re: raw output from copy

From: "Daniel Verite" <daniel(at)manitou-mail(dot)org>
To: "Corey Huinker" <corey(dot)huinker(at)gmail(dot)com>
Cc: "Pavel Stehule" <pavel(dot)stehule(at)gmail(dot)com>,"hlinnaka" <hlinnaka(at)iki(dot)fi>,"PostgreSQL Hackers" <pgsql-hackers(at)postgresql(dot)org>,"Simon Riggs" <simon(at)2ndquadrant(dot)com>,"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>,"Pavel Golub" <pavel(at)microolap(dot)com>,"Craig Ringer" <craig(at)2ndquadrant(dot)com>,"Andrew Dunstan" <andrew(at)dunslane(dot)net>
Subject: Re: raw output from copy
Date: 2016-03-04 14:54:41
Message-ID: e9d94bd7-c7ca-4722-9701-05f12312f5c4@mm
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Corey Huinker wrote:

> So, for me, RAW is the right solution, or at least *a* right solution.

Questions on how to extract from a bytea column come up on a regular
basis, as in [1] [2] [3], or [4] a few days ago, and so far the answers
are to encode the contents in text and decode them in an additional
step, or use COPY BINARY and filter out the headers.

But none of this is as straightforward and efficient as the proposed
COPY RAW.
Also the conversion to text can't be used at all on very large
contents (>512MB), as mentioned in another recent thread [5]
(this is the same reason why pg_dump can't dump such rows),
but COPY RAW doesn't have this limitation.

Technically COPY BINARY should be sufficient, but it seems that
people dislike having to deal with its headers.
Also it's not supported by any of the drivers of popular
script languages that otherwise provide COPY in text format
(DBD::Pg, php, psycopg2...)
Maybe the RAW format would have a better chance to get support
there, because of its simplicity.

[1]
http://www.postgresql.org/message-id/038517CEB6DE43BD8422D7947B6BE8D8@fanliji
ng

[2] http://www.postgresql.org/message-id/4C8272C4.1000008@arcor.de

[3] http://stackoverflow.com/questions/6730729

[4] http://www.postgresql.org/message-id/56C66565.50107@consistentstate.com

[5] http://www.postgresql.org/message-id/14620.1456851036@sss.pgh.pa.us

Best regards,
--
Daniel Vérité
PostgreSQL-powered mailer: http://www.manitou-mail.org
Twitter: @DanielVerite

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2016-03-04 15:08:53 Re: VS 2015 support in src/tools/msvc
Previous Message Michael Paquier 2016-03-04 14:23:54 Re: VS 2015 support in src/tools/msvc