| From: | "Matheus Alcantara" <matheusssilv97(at)gmail(dot)com> |
|---|---|
| To: | "Masahiko Sawada" <sawada(dot)mshk(at)gmail(dot)com> |
| Cc: | "Andrew Dunstan" <andrew(at)dunslane(dot)net>, "jian he" <jian(dot)universality(at)gmail(dot)com>, "Tomas Vondra" <tomas(at)vondra(dot)me>, <pgsql-hackers(at)postgresql(dot)org> |
| Subject: | Re: postgres_fdw: Use COPY to speed up batch inserts |
| Date: | 2026-05-27 23:22:47 |
| Message-ID: | DITUGI3X54S8.346G3XFY7DRDK@gmail.com |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
On Wed Apr 1, 2026 at 12:50 PM -03, Matheus Alcantara wrote:
>> -- on local server
>> copy t(a, d) from stdin;
>> Enter data to be copied followed by a newline.
>> End with a backslash and a period on a line by itself, or an EOF signal.
>>>> 1 hello\nworld
>>>> \.
>> ERROR: invalid input syntax for type integer: "world"
>> CONTEXT: COPY t, line 2, column a: "world"
>> remote SQL command: COPY public.t(a, d) FROM STDIN (FORMAT TEXT)
>>
>
> I think that we need something like CopyAttributeOutText() here.
>
> To fix this I've added appendStringInfoText() which is a similar version
> of CopyAttributeOutText() that works with a StringInfo. I did not find
> any function that I could reuse here, if such function exists please let
> me know.
>
> I'm wondering if we should have this similar function or try to combine
> both to avoid duplicated logic, although it looks complicated to me at
> first look to combine these both usages.
>
I've spent some time hacking into this and actually I don't think that
it's too complicated. Please see the attached patchset.
0001: Extract CopyAttributeOutText escape logic into a reusable function
that can be used by postgres_fdw and by COPY TO routine.
0002: Is the main feature implementation using the TEXT format reusing
the refactor from 0001 avoiding all the duplicated code introduced on
this previous patch version. I've also removed the
resultRelInfo->ri_WithCheckOptions == NIL check that I've introduced
again by mistake since it was already commented at [1] that it should be
removed.
[1] https://www.postgresql.org/message-id/72ec1708-0c02-4ae9-b4f5-ee2bac5fd2f3%40gmail.com
--
Matheus Alcantara
EDB: https://www.enterprisedb.com
| Attachment | Content-Type | Size |
|---|---|---|
| v15-0001-Extract-CopyEscapeText-for-reuse-outside-COPY-TO.patch | text/plain | 6.5 KB |
| v15-0002-postgres_fdw-Use-COPY-as-remote-SQL-when-possibl.patch | text/plain | 18.4 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Matheus Alcantara | 2026-05-27 23:25:12 | Re: postgres_fdw: Use COPY to speed up batch inserts |
| Previous Message | Zsolt Parragi | 2026-05-27 23:10:44 | Re: Heads Up: cirrus-ci is shutting down June 1st |