COPY issue(gsoc project)

From: longlong <asfnuts(at)gmail(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Subject: COPY issue(gsoc project)
Date: 2008-03-11 12:56:47
Message-ID: d9f0a46b0803110556w29795e10p40c9e5439eb4f59d@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

i want to paticipate in gsoc 2008.

here is my plan:
the red part is came from Greg Smith(thanks for Greg Smith's advice).

1.release8.2 make COPY TO can copy the output of an arbitrary SELECT
statement. so i think maybe COPY FROM can get data from output and 'insert
into' some column that designated. the format of the command will be
discussed.

This would be a nice feature. Right now there are often applications
where there is a data loading or staging table that ends up being merged
with a larger table after some cleanup. Moving that data from the
preperation area into the final table right now is most easily done with
INSERT INTO X (SELECT A,B FROM C) type actions. This is slow because
INSERT takes much longer than COPY. Adding support for COPY X FROM
(SELECT A,B FROM C) would make this problem go away.

It is possible to do this right now with some clever use of STDIN/OUT like
the below, but having a pure SQL solution would be more widely applicable.
The overhead of having to pass everything through the client (as STDIN/OUT
do) is certainly not zero.

2.this come from TODO list: COPY always behaviors like a unit of work thar
consists of some insert commands, if any error, it rollback. but sometimes
we only care the data should be inserted. in that situation, i used to use
"try....catch...." insert row by row to skip the error, because it will take
much time to examine every row. so:
Allow COPY to report error lines and continue.
this is a good idea.

This is a long standing request and many people would be happy to see it
implemented. You do want to make sure the implementation easily allows
pushing all the lines that didn't commit into what's commonly called a
"reject file".

----------------------------------------------------------------------------

is these feasible?
which one should i choose to proposal or both?

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2008-03-11 13:00:36 Re: strange pg_ctl's behavior
Previous Message KaiGai Kohei 2008-03-11 12:56:09 [ANN] Now v8.3 based SE-PostgreSQL is available