Re: Different length lines in COPY CSV

Tom Lane wrote:

>>Where would you want this file conversion
>>utility? bin? contrib? pgfoundry?
>>
>>
>
>I'd say pgfoundry for starters --- there's no reason to tie it down
>to server release cycles. Maybe when the thing is fairly mature and
>doesn't need frequent releases, we could think about whether it ought
>to be brought into the core distro.
>
>However, it likely won't ever be a candidate to become part of core
>unless it's written in C, and offhand I would judge C to not be the
>best choice of implementation language for such a thing. This is surely
>going to be mostly a string-pushing type of problem, so something like
>perl might be a better bet.
>
>
>
>

You are probably right. The biggest wrinkle will be dealing with various
encodings, I suspect. That at least is one thing that doing CSV within
the backend bought us fairly painlessly. Perl's Text::CSV_XS module for
example simply handles this by declaring that only [\x09\x20-\x7f] are
valid in its non-binary mode, and in either mode appears to be MBCS
unaware. We should try to do better than that.

cheers

andrew