Re: is it possible to flush / commit / update underlying table during the COPY table FROM 'pipe'

From: Jon Jensen <jon(at)jenseng(dot)com>
To: Trent George <trent(dot)george1(at)gmail(dot)com>
Cc: pgsql-novice(at)postgresql(dot)org
Subject: Re: is it possible to flush / commit / update underlying table during the COPY table FROM 'pipe'
Date: 2011-03-27 18:09:12
Message-ID: CB451BCE-8AC5-46F3-BAF4-CF0314C305C0@jenseng.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-novice

On Mar 26, 2011, at 5:31 PM, Trent George wrote:

> Is it (or would it) be possible to allow some type of equivalent of
> BATCHSIZE as used in mssql BULK INSERT command
> This would group and commit this number of records during the import,
> while it is still running
> One could in theory use a value of =1, =1000 or leave the default (whole batch)

There is nothing built-in for what you describe, but you could accomplish the desired result on your end. e.g. split the import into batches of 1000 (or whatever) and issue individual COPY commands for each one. This could all be wrapped into a sh/perl/whatever script to make it reusable, configurable, add error handling, etc.

Jon

In response to

Browse pgsql-novice by date

  From Date Subject
Next Message Francisco Leovey 2011-03-27 18:53:51 Re: /etc/init.d/postgres-8.4-openscg start problem
Previous Message Trent George 2011-03-26 23:31:14 is it possible to flush / commit / update underlying table during the COPY table FROM 'pipe'