Re: multiline CSV fields

From: Bruce Momjian <pgman(at)candle(dot)pha(dot)pa(dot)us>
To: Andrew Dunstan <andrew(at)dunslane(dot)net>
Cc: Greg Stark <gsstark(at)mit(dot)edu>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: multiline CSV fields
Date: 2004-11-30 17:57:58
Message-ID: 200411301757.iAUHvwU22351@candle.pha.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers pgsql-patches

Andrew Dunstan wrote:
>
>
> Greg Stark wrote:
>
> >Personally I find the current CSV support inadequate. It seems pointless to
> >support CSV if it can't load data exported from Excel, which seems like the
> >main use case.
> >
> >
>
> OK, I'm starting to get mildly annoyed now. We have identified one
> failure case connected with multiline fields. Even in the multiline
> case, I expect that the embedded newlines will usually match those of
> the CSV file, in which case there will be no failure. It's a very big
> step from there to the far more general "can't load data exported from
> Excel". Or did you have some other limitation in mind?
>
> FWIW, I don't make a habit of using multiline fields in my spreadsheets
> - and some users I have spoken to aren't even aware that you can have
> them at all.

I am wondering if one good solution would be to pre-process the input
stream in copy.c to convert newline to \n and carriage return to \r and
double data backslashes and tell copy.c to interpret those like it does
for normal text COPY files. That way, the changes to copy.c might be
minimal; basically, place a filter in front of the CSV file that cleans
up the input so it can be more easily processed.

--
Bruce Momjian | http://candle.pha.pa.us
pgman(at)candle(dot)pha(dot)pa(dot)us | (610) 359-1001
+ If your life is a hard drive, | 13 Roberts Road
+ Christ can be your backup. | Newtown Square, Pennsylvania 19073

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2004-11-30 18:02:58 Re: [Testperf-general] Re: 8.0beta5 results w/ dbt2
Previous Message Tom Lane 2004-11-30 17:10:34 Re: 8.0beta5 results w/ dbt2

Browse pgsql-patches by date

  From Date Subject
Next Message Bruce Momjian 2004-11-30 19:00:59 Re: [HACKERS] psql \e broken again
Previous Message Alvaro Herrera 2004-11-30 16:38:53 Re: Charset WIN1252