Re: Syntax error in a large COPY

From: Reg Me Please <regmeplease(at)gmail(dot)com>
To: "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com>
Cc: pgsql-general(at)postgresql(dot)org, "Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us>
Subject: Re: Syntax error in a large COPY
Date: 2007-11-06 21:51:05
Message-ID: 200711062251.12812.regmeplease@gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Il Tuesday 06 November 2007 22:37:12 Scott Marlowe ha scritto:
> On 11/6/07, Reg Me Please <regmeplease(at)gmail(dot)com> wrote:
> > Il Tuesday 06 November 2007 22:13:15 hai scritto:
> > > On 11/6/07, Reg Me Please <regmeplease(at)gmail(dot)com> wrote:
> > > > Il Tuesday 06 November 2007 19:43:38 Scott Marlowe ha scritto:
> > > > > On 11/6/07, Reg Me Please <regmeplease(at)gmail(dot)com> wrote:
> > > > > > That seems not to be the case.
> > > > > > The last line has a \. by its own and the last but one is
> > > > > > well formed.
> > > > >
> > > > > (Please don't top post...)
> > > > >
> > > > > Got a self contained test case you can post?
> > > >
> > > > Back to the original topic ...
> > > >
> > > > I'm trying to understand what and where.
> > > > The point is that I have this 29M+ lines script telling me there's a
> > > > problem somewhere.
> > > >
> > > > A self contained test, at the moment, would be that long!
> > > >
> > > > I'm considering a "branch and bound" approach ... but it'd be quite
> > > > long and tedious as the program generating the script has not been
> > > > written to do such things.
> > >
> > > Split it in half, with the appropriate sql on each end so the data
> > > still works, and see which half causes a problem. Keep splitting the
> > > one that causes a problem in half until you have a small one with the
> > > problem still.
> > >
> > > I'm guessing the problem will become obvious then.
> >
> > That's the "branch and bound". Editing 29M+ lines file takes some time.
> > But this is the way I'm going to go right now.
>
> Oh, we called it half-splitting in the military.
>
> Using something like head / tail in unix to do it. Should be fairly
> fast, especially if you keep cutting it in half after the first test.
> I can't imagine editing something that large even in vi being very
> fast.

My laptop has a fairly slow disk and even with Linux and vi it takes time.

Well, a better diagnostic messages would be better in any case.

--
Reg me Please

In response to

Browse pgsql-general by date

  From Date Subject
Next Message Thomas Kellerer 2007-11-06 22:05:18 Re: Syntax error in a large COPY
Previous Message Tim Goodaire 2007-11-06 21:43:23 Re: subversion support?