Re: COPY FROM : out of memory

From: Arnaud Lesauvage <thewild(at)freesurf(dot)fr>
To: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Arnaud Lesauvage <thewild(at)freesurf(dot)fr>, pgsql-general(at)postgresql(dot)org
Subject: Re: COPY FROM : out of memory
Date: 2006-11-23 16:57:58
Message-ID: 4565D316.8090600@freesurf.fr
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Martijn van Oosterhout a écrit :
> On Thu, Nov 23, 2006 at 11:27:06AM -0500, Tom Lane wrote:
>> Arnaud Lesauvage <thewild(at)freesurf(dot)fr> writes:
>> > When trying to import a 20M rows csv file into PostgreSQL, I
>> > get :
>>
>> > ERROR: out of memory
>> > État SQL :53200
>> > Détail :Failed on request of size 1073741823.
>> > Contexte : COPY tmp, line 1
>>
>> Can you put together a self-contained example? The reference to "line
>> 1" suggests that you wouldn't need the whole 20M row file, just the
>> first few rows ...
>
> Maybe it's a line termination problem?
>
> Have a nice day,

I think you are right !
Trying to see the first line with sed outputs the whole file!
All I did was export the file in UNICODE from MSSQL, convert
it with iconv -f "UCS-4-INTERNAL" -t "UTF-8" myfile.cvs.

I guess I still don't have the right encoding... :(

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Arnaud Lesauvage 2006-11-23 17:05:02 Re: MSSQL to PostgreSQL : Encoding problem
Previous Message Richard Broersma Jr 2006-11-23 16:57:23 Re: Why overlaps is not working