Skip site navigation (1) Skip section navigation (2)

Re: Fwd: I would like to alter the COPY command

From: Andrew Dunstan <andrew(at)dunslane(dot)net>
To: Mason <freemason(at)gmail(dot)com>
Cc: pgsql-patches(at)postgresql(dot)org
Subject: Re: Fwd: I would like to alter the COPY command
Date: 2006-12-29 17:43:09
Message-ID: 459553AD.3010806@dunslane.net (view raw or flat)
Thread:
Lists: pgsql-patchespgsql-sql
Mason wrote:
> What I have is data with two different characters for "start quote"
> and "end quote".  In my case it's '[' and ']', but it could be
> anything from "smart quotes", to parentheses, to brackets, braces, ^/$
> in regexps, etc.  I think this isn't too unreasonable a feature to
> have to make copy more functional when importing data that is
> difficult to transform properly beforehand (in my case is about half a
> terabyte of log files, which takes hours and hours, just to cat, let
> alone reparse and dump into COPY).

I think regexps would be going too far. One simple approach which 
wouldn't involve any grammar changes would be to allow the quote 
parameter to be 2 chars long instead of one, and treat the second as the 
closing quote char (which would otherwise default to the first char). An 
argument against that would be that it would preclude us from in future 
allowing multi-char quote marks. I'm not sure if that matters quite so 
much as it would for the delimiter parameter.

>
> Now, in my case I can just say "cat file | tr '[]' '""' | psql -f
> import.sql", but then I lose the ability for psql to do anything smart
> like using mmap (I'm making assumptions that it does anything smart
> like that, but even if it doesn't now, it could some day).
>

Well, you also earn a "Useless use of cat" award ;-)

And no, we don't mmap the file, but we do do some smart buffering stuff, 
thanks to Greenplum.

cheers

andrew

In response to

pgsql-patches by date

Next:From: Roman KononovDate: 2006-12-29 19:54:41
Subject: Re: [HACKERS] [BUGS] BUG #2846: inconsistent and confusing
Previous:From: Stefan KaltenbrunnerDate: 2006-12-29 17:32:44
Subject: Re: Recent SIGSEGV failures in buildfarm HEAD

pgsql-sql by date

Next:From: Bruce MomjianDate: 2006-12-30 02:09:49
Subject: Re: domains and serial
Previous:From: RagnarDate: 2006-12-29 17:18:41
Subject: Re: How to reduce a database

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group