Skip site navigation (1) Skip section navigation (2)

Re: ERROR: invalid memory alloc request size

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Alvaro Herrera <alvherre(at)commandprompt(dot)com>
Cc: Johann Spies <jspies(at)sun(dot)ac(dot)za>, pgsql-admin(at)postgresql(dot)org
Subject: Re: ERROR: invalid memory alloc request size
Date: 2007-05-28 15:18:37
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-admin
Alvaro Herrera <alvherre(at)commandprompt(dot)com> writes:
> Johann Spies wrote:
>> #3  0x00000000005ff1c8 in perform_default_encoding_conversion (
>> src=0x2b881e87f040 "\"2007-04-20\" \"18:07:06\" \"\" \"73\" \"\\\"\"\n\"2007-04-20\" \"18:07:06\" \"\" \"69\" \"\"\n\"20"..., len=337739539, is_client_to_server=5 '\005') at mbutils.c:461

> Interesting.  You are passing a single 300 MB string here, not
> 90-something char lines.  The problem is probably in newlines.  Why are
> they not getting used as line separators escapes me.

Stripped of gdb's own backslashing, the data seems to be 


which leads me to think that the dump was generated on the assumption
that backslash is the escape character, but is being reloaded under a
different assumption.  Once you got out-of-sync on that one data value,
you'd have a mess --- all the intended line-ending newlines would be
inside quotes, up till you hit another URL with an embedded quote
(or newline?).  CSV is not exactly a robust representation :-(

So the short answer seems to be that we were all misled by the large
alloc-request number into thinking there was some data-corruption
behavior involved, when we should have been quizzing Johann on exactly
what COPY parameters he was giving.

			regards, tom lane

In response to


pgsql-admin by date

Next:From: Alexander B.Date: 2007-05-28 20:06:55
Subject: Return record on recursive call
Previous:From: Alvaro HerreraDate: 2007-05-28 14:46:51
Subject: Re: ERROR: invalid memory alloc request size

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group