| From: | "Andrey V(dot) Lepikhov" <a(dot)lepikhov(at)postgrespro(dot)ru> |
|---|---|
| To: | "tsunakawa(dot)takay(at)fujitsu(dot)com" <tsunakawa(dot)takay(at)fujitsu(dot)com> |
| Cc: | "tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com" <tanghy(dot)fnst(at)cn(dot)fujitsu(dot)com>, Alexey Kondratov <a(dot)kondratov(at)postgrespro(dot)ru>, Michael Paquier <michael(at)paquier(dot)xyz>, Ashutosh Bapat <ashutosh(dot)bapat(at)2ndquadrant(dot)com>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>, PostgreSQL-Dev <pgsql-hackers(at)postgresql(dot)org>, Tomas Vondra <tomas(dot)vondra(at)enterprisedb(dot)com>, "houzj(dot)fnst(at)cn(dot)fujitsu(dot)com" <houzj(dot)fnst(at)cn(dot)fujitsu(dot)com>, Amit Langote <amitlangote09(at)gmail(dot)com> |
| Subject: | Re: [POC] Fast COPY FROM command for the table with foreign partitions |
| Date: | 2021-02-09 05:57:44 |
| Message-ID: | 737f24c8-3f0c-4201-e3da-b763dbf9c8b9@postgrespro.ru |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On 2/9/21 9:35 AM, tsunakawa(dot)takay(at)fujitsu(dot)com wrote:
> From: tsunakawa(dot)takay(at)fujitsu(dot)com <tsunakawa(dot)takay(at)fujitsu(dot)com>
>> From: Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>
>> Also, I might defer working on the extended part (v9 0003 and 0004) and further
>> separate them in a different thread, if it seems to take longer.
>
> I reviewed them but haven't rebased them (it seems to take more labor.)
> Andrey-san, could you tell us:
>
> * Why is a separate FDW connection established for each COPY? To avoid using the same FDW connection for multiple foreign table partitions in a single COPY run?
With separate connection you can init a 'COPY FROM' session for each
foreign partition just one time on partition initialization.
>
> * In what kind of test did you get 2-4x performance gain? COPY into many foreign table partitions where the input rows are ordered randomly enough that many rows don't accumulate in the COPY buffer?
I used 'INSERT INTO .. SELECT * FROM generate_series(1, N)' to generate
test data and HASH partitioning to avoid skews.
--
regards,
Andrey Lepikhov
Postgres Professional
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Ashutosh Bapat | 2021-02-09 05:59:14 | Re: repeated decoding of prepared transactions |
| Previous Message | Kyotaro Horiguchi | 2021-02-09 05:55:23 | Re: Is Recovery actually paused? |