From: | Michael Paquier <michael(dot)paquier(at)gmail(dot)com> |
---|---|
To: | Bruce Momjian <bruce(at)momjian(dot)us> |
Cc: | Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Daniel Verite <daniel(at)manitou-mail(dot)org>, Robert Haas <robertmhaas(at)gmail(dot)com>, Jim Nasby <Jim(dot)Nasby(at)bluetreble(dot)com>, Ronan Dunklau <ronan(dot)dunklau(at)dalibo(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org> |
Subject: | Re: pg_dump / copy bugs with "big lines" ? |
Date: | 2016-04-25 07:35:33 |
Message-ID: | CAB7nPqSvHz_ECbQPPCrEpfFSQNpYnu=TpauZ4cAk+mLeWxTdoA@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Sat, Apr 23, 2016 at 9:58 AM, Bruce Momjian <bruce(at)momjian(dot)us> wrote:
> On Thu, Mar 3, 2016 at 10:31:26AM +0900, Michael Paquier wrote:
>> On Thu, Mar 3, 2016 at 12:47 AM, Alvaro Herrera
>> <alvherre(at)2ndquadrant(dot)com> wrote:
>> > Well, the CopyData message has an Int32 field for the message length.
>> > I don't know the FE/BE protocol very well but I suppose each row
>> > corresponds to one CopyData message, or perhaps each column corresponds
>> > to one CopyData message. In either case, it's not possible to go beyond
>> > 2GB without changing the protocol ...
>>
>> Based on what I know from this stuff (OOM libpq and other stuff
>> remnants), one 'd' message means one row. fe-protocol3.c and
>> CopySendEndOfRow in backend's copy.c are confirming that as well. I am
>> indeed afraid that having extra logic to get chunks of data will
>> require extending the protocol with a new message type for this
>> purpose.
>
> Is there any documentation that needs updating based on this research?
Perhaps. On the docs the two sections referring to the CopyData
messages arein protocol.sgml
- with this portion for the 'd' message itself:
<term>
CopyData (F & B)
</term>
- and a more precise description here:
<sect2 id="protocol-copy">
<title>COPY Operations</title>
We could precise in one of them that the maximum size of a CopyData
message can be up to 1GB. Thoughts?
--
Michael
From | Date | Subject | |
---|---|---|---|
Next Message | Magnus Hagander | 2016-04-25 10:17:56 | Re: Rename max_parallel_degree? |
Previous Message | Michael Paquier | 2016-04-25 07:11:09 | Re: VS 2015 support in src/tools/msvc |