Re: COPY Transform support

From: PFC <lists(at)peufeu(dot)com>
To: "Dimitri Fontaine" <dfontaine(at)hi-media(dot)com>
Cc: NikhilS <nikkhils(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: COPY Transform support
Date: 2008-04-03 15:22:28
Message-ID: op.t81jnqu7cigqcu@apollo13.peufeu.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>> INSERT INTO mytable (id, date, ...) SELECT id, NULLIF( date,
>> '0000-00-00' ), ... FROM mydump WHERE (FKs check and drop the borken
>> records);
>
> What do we gain against current way of doing it, which is:
> COPY loadtable FROM 'dump.txt' WITH ...
> INSERT INTO destination_table(...) SELECT ... FROM loadtable;

You read and write the data only once instead of twice (faster) if you
want to import all of it.
If you just want to compute some aggregates and store the results in a
table, you just read the data once and don't write it at all.

The advantages are the same than your proposed transformations to COPY,
except I feel this way of doing it opens more options (like, you can
combine columns, check FKs at load, do queries on data without loading it,
don't necessarily have to insert the data in a table, don't have to invent
a new syntax to express the transformations, etc).

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Mark Mielke 2008-04-03 15:32:37 Re: [GENERAL] SHA1 on postgres 8.3
Previous Message PFC 2008-04-03 15:15:35 Re: COPY Transform support