Re: Copy command-Out of memory error

On Tue, Oct 09, 2012 at 03:03:39PM -0400, Nipuna wrote:
> Hi Tom,
>
> Thanks for the response.I did retrieve that particular line to see if that
> record is wider, but its the same as other records. Anymore suggestions..
>

Did you check for quoting or escaping problems as Tom suggested, as well.

Cheers,
Ken

> On Tue, Oct 9, 2012 at 2:43 PM, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
>
> > nipuna <nipunajoset(at)gmail(dot)com> writes:
> > > I am trying to copy a 13G file to postgres db using the 'copy from'
> > command on PGSQL 8.4. After loading 8 million records I got the below error
> > .
> >
> > > ERROR: out of memory
> >
> > > DETAIL: Cannot enlarge string buffer containing 1073726116 bytes by
> > 65536 more bytes.
> > > CONTEXT: COPY teststorage, line 8479575
> >
> > This makes it look like line 8479575 is, all by itself, wider than 1GB.
> > If that's really what your data is, you'll need to find some other
> > representation --- Postgres can't cope with fields wider than 1GB.
> > (And in practice you don't want to be coming anywhere close to that
> > limit anyway.)
> >
> > If you didn't think you had individual rows that wide, maybe this is
> > due to some sort of quoting or escaping problem in your data file.
> >
> > regards, tom lane
> >
>
>
>
> --
> Regards,
> Nipuna