Re: How to implement the skip errors for copy from ?

Use subtraction is very inefficient, a project called pg_bulkload support the?skip errors ,and it does not useing subtraction. It?performance is very good.??So I want to imitate?pg_bulkload?to?implementation?skip errors of copy.if i do the following thing to copy :1. disable all of trigger of table;2. only skip the following errors:? ? ?* format error of tuple;? ? ?* check constraint?violation;? ? ?* unique or primary key?constraint?violation;? ? other errors will be abort current transcation except above three errors.3. ?bad tuple will be deleted and the?resource owner per tuples is reseased?? ? when?a unique key?violation, ?xmax of the tuple will?be marked to current? ? transaction id as it?not be seen as good, and all of index?entries?of the bad? ? tuple will?be real delete at next VACUUM.Is it right to?skip errors for copy from?

From:?Tom LaneDate:?2014-06-17?21:40To:?xbzhangCC:?Amit Kapila; Pavel Stehule; Alvaro Herrera; pgsql-hackersSubject:?Re: [HACKERS] How to implement the skip errors for copy from ?xbzhang <xbzhang(at)kingbase(dot)com(dot)cn> writes:
> LWlocks can record in resource owner per tuples, so they can be releasedat rigth way, but the memory allocated on memory contextis one problem.Are there any others problems?
?
See AbortSubTransaction(), CleanupSubTransaction(), and the rather large
number of subroutines they call.? Almost everything that code does is
connected to cleaning up something that might have been left unfinished
after an elog(ERROR) took control away in the middle of some code
sequence.
?
In addition, you can't just wave your hands and presto the bad tuple is
not there anymore.? For example, the failure might have been a unique key
violation in some index or other.? Not only is the bad tuple already on
disk, but possibly so are index entries for it in other indexes.? In
general the only way to get rid of those index entries is a VACUUM.
So you really have to have a subtransaction whose XID is what you mark
the new tuple with, and then rolling back the subtransaction is what
causes the new tuple to not be seen as good.? (Actually getting rid of
it will be left for the next VACUUM.)
?
regards, tom lane
?
?
?
?
-----
???????????
????:AVG - www.avg.com
??:2013.0.3480 / ?????:3955/7691 - ????:06/17/14
?