Re: [POC] Fast COPY FROM command for the table with foreign partitions

From: "Andrey V(dot) Lepikhov" <a(dot)lepikhov(at)postgrespro(dot)ru>
To: Ashutosh Bapat <ashutosh(dot)bapat(at)2ndquadrant(dot)com>
Cc: Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>, PostgreSQL-Dev <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [POC] Fast COPY FROM command for the table with foreign partitions
Date: 2020-07-03 05:15:46
Message-ID: 5de28d65-3c43-e460-f834-fabbfe8233a4@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 6/22/20 5:11 PM, Ashutosh Bapat wrote:

> <a(dot)lepikhov(at)postgrespro(dot)ru <mailto:a(dot)lepikhov(at)postgrespro(dot)ru>> wrote:
> It looks like we call BeginForeignInsert and EndForeignInsert even
> though actual copy is performed using BeginForeignCopy, ExecForeignCopy
> and EndForeignCopy. BeginForeignInsert constructs the INSERT query which
> looks unnecessary. Also some of the other PgFdwModifyState members are
> initialized unnecessarily. It also gives an impression that we are using
> INSERT underneath the copy. Instead a better way would be to
> call BeginForeignCopy instead of BeginForeignInsert and EndForeignCopy
> instead of EndForeignInsert, if we are going to use COPY protocol to
> copy data to the foreign server. Corresponding postgres_fdw
> implementations need to change in order to do that.
>
I did not answer for a long time, because of waiting for the results of
the discussion on Tomas approach to bulk INSERT/UPDATE/DELETE. It seems
more general.
I can move the query construction into the first execution of INSERT or
COPY operation. But another changes seems more invasive because
BeginForeignInsert/EndForeignInsert are used in the execPartition.c
module. We will need to pass copy/insert state of operation into
ExecFindPartition() and ExecCleanupTupleRouting().

--
regards,
Andrey Lepikhov
Postgres Professional

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Peter Eisentraut 2020-07-03 06:17:45 Re: POC: rational number type (fractions)
Previous Message Michael Paquier 2020-07-03 05:10:59 Re: A patch for get origin from commit_ts.