Re: importing data from Filemaker: weird newline characters

On Mon, Jan 21, 2002 at 05:21:17PM -0600, Ross J. Reedstrom wrote:
> On Mon, Jan 21, 2002 at 05:41:03PM +0100, Frank Joerdens wrote:
> > After importing a table from Filemaker 5.5 (via COPY from a csv file),
> > I've got newline characters from text fields in the original Filemaker
> > db which appear as ^K, or control-K on the console in psql or vi. I am
> > wondering how to deal with those. How do I find out what it is? Are
> > there standards as to how to represent newlines etc. in text fields in
> > SQL databases?
>
> Low, many suns ago, I also imported some text from Filemaker: that's how
> FM exports newlines. Postgresql can handle newlines in it's data, by
> quoting them with '\'. So a sed run on the CSV file with something like:
>
> sed 's/^K/\\^M' <old.csv >new.csv
>
> Where the ^K and ^M are actual control codes (generated in bash with
> ^I^K and ^I^M) should do it.

Hm . . . I think I know what you mean, in that I get the idea. I feel
kinda stupid asking again since I feel I should somehow know this: No
searching in my O'Reilly sed & awk book, no permutations of your example
on my command line, and no Google (I looked at the Advanced
Bash-Scripting Guide and the Text-Terminal-HOWTO) managed to resolve the
riddle of how to generate 'actual control codes'. Just typing

sed 's/\^K/\^M/' < adressen.csv > new_adressen.csv
^ ^ ^
(this looks a little different to your line - I think it's what you
"meant" - your line gives a syntax error with sed because of the missing
/ after ^M)

obviously doesn't work. I'm saying obviously because you are already
saying, in effect, that that's not what you mean. It's not really all
that obvious to me because . . . well, I guess it's a real gap in my
understanding of how the shell deals these with special characters . . .
and I seem to be having real difficulties in finding a resource that
would help me close it. Can you try to explain again how I might
convince sed to perform the substitution? Or even where I could read up
on the matter? (If I can't get sed to do it - which would be rather neat
- I'd probably try to find a Hex editor next, and try to figure out what
^K is there.)

Regards, Frank