Skip site navigation (1) Skip section navigation (2)

Re: ETL optimization

From: Bricklen Anderson <BAnderson(at)PresiNET(dot)com>
To: Dennis Bjorklund <db(at)zigo(dot)dhs(dot)org>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: ETL optimization
Date: 2005-06-27 18:15:06
Message-ID: 42C0422A.7020408@PresiNET.com (view raw or flat)
Thread:
Lists: pgsql-performance
Dennis Bjorklund wrote:
> On Thu, 23 Jun 2005, Bricklen Anderson wrote:
> 
> 
>>iii. UNIQUE constraint on table "t1". This didn't seem to perform too
>>badly with fewer rows (preliminary tests), but as you'd expect, on error
>>the whole transaction would roll back. Is it possible to skip a row if
>>it causes an error, as opposed to aborting the transaction altogether?
> 
> 
> You don't need to roll back the whole transaction if you use savepoints or 
> the exception features in pl/pgsql
> 
> Take a look at this example:
> 
> http://developer.postgresql.org/docs/postgres/plpgsql-control-structures.html#PLPGSQL-UPSERT-EXAMPLE
> 
Hmmm... forgot about savepoints. That's an interesting idea that I'll have to
check out. I earlier mentioned that I was going to test the delete + insert
version, and it works pretty well. I got it down to about 3 minutes using that
method. I'll test the savepoint and the exception version that you listed as well.

Thanks!

-- 
_______________________________

This e-mail may be privileged and/or confidential, and the sender does
not waive any related rights and obligations. Any distribution, use or
copying of this e-mail or the information it contains by other than an
intended recipient is unauthorized. If you received this e-mail in
error, please advise me (by return e-mail or otherwise) immediately.
_______________________________

In response to

pgsql-performance by date

Next:From: Luke LonerganDate: 2005-06-27 18:52:30
Subject: Re: COPY FROM performance improvements
Previous:From: Merlin MoncureDate: 2005-06-27 18:14:10
Subject: Re: [HACKERS] How two perform TPC-H test on postgresql-8.0.2

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group