From: | "Scott Marlowe" <scott(dot)marlowe(at)gmail(dot)com> |
---|---|
To: | Seb <spluque(at)gmail(dot)com> |
Cc: | pgsql-general(at)postgresql(dot)org |
Subject: | Re: inserting only new rows from csv file |
Date: | 2008-10-01 21:38:57 |
Message-ID: | dcc563d10810011438i7039729bm4779835412d35e56@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-general |
On Wed, Oct 1, 2008 at 3:16 PM, Seb <spluque(at)gmail(dot)com> wrote:
> Hi,
>
> I've seen some approaches to this problem, but I suspect there are
> better ways.
>
> A csv file is produced nightly, with the same table structure as a table
> in a PostgreSQL database. The file grows over time as new records are
> added every day. However, the rows in the file may not be sorted the
> same way every day.
>
> I need to isolate the new rows and insert them into the database table.
> Before I start writing a script to do this, I need to decide whether to
> do the comparison between the database table and the csv file: a) within
> the database, or b) COPY'ing the table to a file and then do the
> comparison. I'm not very experienced with SQL, so am more comfortable
> with the latter option using plain shell and awk. Which route is best?
> If a), I'd appreciate some SQL code to do it. Thanks.
I'd load them into a staging table, and use a query (either a
subselect or a left join where null type query) to load them into the
master table.
From | Date | Subject | |
---|---|---|---|
Next Message | Ivan Sergio Borgonovo | 2008-10-01 22:03:45 | Re: "object references" and renaming was: Why Does UPDATE Take So Long? |
Previous Message | Dennis Brakhane | 2008-10-01 21:20:47 | Re: Transactions within a function body |