Re: foreign table batch inserts

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Manuel Kniep <m(dot)kniep(at)web(dot)de>
Cc: PostgreSQL mailing lists <pgsql-hackers(at)postgresql(dot)org>, fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp
Subject: Re: foreign table batch inserts
Date: 2016-05-17 22:08:28
Message-ID: CAB7nPqRt7xtUaX0s3XXGELdtTEFn7xME=fdAFcaBCo80quBroA@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, May 18, 2016 at 6:00 AM, Manuel Kniep <m(dot)kniep(at)web(dot)de> wrote:
> I realized that inserts into foreign tables are only done row by row.
> Consider copying data from one local table to a foreign table with
>
> INSERT INTO foreign_table(a,b,c) SELECT a,b,c FROM local_table;
>
> When the foreign server is for example in another datacenter with long latency,
> this as an enormous performance trade off.
>
> Wouldn’t it make sense to do the insert batch wise e.g. 100 rows ?

Using a single query string with multiple values, perhaps, but after
that comes into consideration query string limit particularly for
large text values... The query used for the insertion is a prepared
statement since writable queries are supported in 9.3, which makes the
code quite simple actually.

> Are there any plans doing that or am I miss something?

Not that I know of. I am adding Fujita-san in the loop here, he is
quite involved with postgres_fdw these days so perhaps he has some
input to offer.
--
Michael

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Geoghegan 2016-05-17 22:33:45 Re: Parallel query and temp_file_limit
Previous Message Tom Lane 2016-05-17 21:48:57 Re: Re: [COMMITTERS] pgsql: Allocate all page images at once in generic wal interface