pgloader an Indexes / was: Re: CSV-bulk import and defaults

From: Thomas Schmidt <postgres(at)stephan(dot)homeunix(dot)net>
To: pgsql-general(at)postgresql(dot)org
Subject: pgloader an Indexes / was: Re: CSV-bulk import and defaults
Date: 2011-01-03 08:48:22
Message-ID: 4D218D56.6080405@stephan.homeunix.net
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

Hello,

Am 03.01.11 00:06, schrieb Adrian Klaver:
> On Sunday 02 January 2011 2:22:14 pm Thomas Schmidt wrote:
>> well, I'm new to postgres and this is my post on this list :-)
>> Anyway, I've to batch-import bulk-csv data into a staging database (as
>> part of an ETL-"like" pocess). The data ought to be read via STDIN,
>> however for keeping in simple and stupid, saving it to a file and
>> importing afterwards is also an option. Sticking my nose into the docs,
>> I noticed that copy[1] as well as pg_import[2] are able to do it.
>>
>> However, there are some additional columns of the staging table (job id,
>> etc.) that have to be set in order to identify imported rows. These
>> attributes are not part of the data coming from STDIN (since its
>> meta-data) and I see no way for specifying default values for "missing"
>> cvs columns. (imho copy and pg_bulkload will use table defaults for
>> missing rows - do I miss something?).
>>
>> [1] http://www.postgresql.org/docs/9.0/static/sql-copy.html
>> [2] http://pgbulkload.projects.postgresql.org/pg_bulkload.html
> Check out pgloader:
> http://pgloader.projects.postgresql.org/

Thanks a lot - that's what I need. :-)
Btw. What about indexes?
http://www.postgresql.org/docs/9.0/interactive/populate.html suggests to
remove indexes before importing via copy (for obvious reasons).
Does pgloader take indexes into account or do I need to handle 'em manually?

Thanks in adance,
Thomas

In response to

Responses

Browse pgsql-general by date

  From Date Subject
Next Message Andre Lopes 2011-01-03 11:11:56 Need advise for database structure for non linear data.
Previous Message Alban Hertroys 2011-01-03 07:24:03 Re: problem updating from form