Re: Fast COPY FROM based on batch insert

From: Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
To: Ian Barwick <ian(dot)barwick(at)enterprisedb(dot)com>
Cc: Justin Pryzby <pryzby(at)telsasoft(dot)com>, Zhihong Yu <zyu(at)yugabyte(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com>, tanghy(dot)fnst(at)fujitsu(dot)com, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>, houzj(dot)fnst(at)fujitsu(dot)com
Subject: Re: Fast COPY FROM based on batch insert
Date: 2022-07-11 03:54:08
Message-ID: dd6f005d-6773-9335-ccba-8ff55db626a4@postgrespro.ru
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 11/7/2022 04:12, Ian Barwick wrote:
> On 09/07/2022 00:09, Andrey Lepikhov wrote:
>> On 8/7/2022 05:12, Ian Barwick wrote:
>>>      ERROR:  bind message supplies 0 parameters, but prepared
>>> statement "pgsql_fdw_prep_178" requires 6
>>>      CONTEXT:  remote SQL command: INSERT INTO public.foo_part_1(t,
>>> v1, v2, v3, v4, v5) VALUES ($1, $2, $3, $4, $5, $6)
>>>      COPY foo, line 88160
>> Thanks, I got it. MultiInsertBuffer are created on the first non-zero
>> flush of tuples into the partition and isn't deleted from the buffers
>> list until the end of COPY. And on a subsequent flush in the case of
>> empty buffer we catch the error.
>> Your fix is correct, but I want to propose slightly different change
>> (see in attachment).
>
> LGTM.
New version (with aforementioned changes) is attached.

--
regards,
Andrey Lepikhov
Postgres Professional

Attachment Content-Type Size
v4-0001-Implementation-of-a-Bulk-COPY-FROM.patch text/plain 23.1 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message merryok 2022-07-11 03:58:06 wal write of large object
Previous Message Joseph D Wagner 2022-07-11 03:45:38 proposal: Allocate work_mem From Pool