|From:||Andrey Lepikhov <a(dot)lepikhov(at)postgrespro(dot)ru>|
|To:||Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>|
|Subject:||Re: [POC] Fast COPY FROM command for the table with foreign partitions|
|Views:||Raw Message | Whole Thread | Download mbox | Resend email|
19.06.2020 19:58, Etsuro Fujita пишет:
> On Tue, Jun 2, 2020 at 2:51 PM Andrey Lepikhov
> <a(dot)lepikhov(at)postgrespro(dot)ru> wrote:
>> Hiding the COPY code under the buffers management machinery allows us to
>> generalize buffers machinery, execute one COPY operation on each buffer
>> and simplify error handling.
> I'm not sure that it's really a good idea that the bulk-insert API is
> designed the way it's tightly coupled with the bulk-insert machinery
> in the core, because 1) some FDWs might want to send tuples provided
> by the core to the remote, one by one, without storing them in a
> buffer, or 2) some other FDWs might want to store the tuples in the
> buffer and send them in a lump as postgres_fdw in the proposed patch
> but might want to do so independently of MAX_BUFFERED_TUPLES and/or
> MAX_BUFFERED_BYTES defined in the bulk-insert machinery.
> I agree that we would need special handling for cases you mentioned
> above if we design this API based on something like the idea I
> proposed in that thread.
>> As i understand, main idea of the thread, mentioned by you, is to add
>> "COPY FROM" support without changes in FDW API.
> I don't think so; I think we should introduce new API for this feature
> to keep the ExecForeignInsert() API simple.
>> All that I can offer in this place now is to introduce one new
>> ExecForeignBulkInsert(buf) routine that will execute single "COPY FROM
>> STDIN" operation, send tuples and close the operation. We can use the
>> ExecForeignInsert() routine for each buffer tuple if
>> ExecForeignBulkInsert() is not supported.
In the next version (see attachment) of the patch i removed Begin/End
fdwapi routines. Now we have only the ExecForeignBulkInsert() routine.
|Next Message||Dilip Kumar||2020-06-22 06:25:49||Re: PATCH: logical_work_mem and logical streaming of large in-progress transactions|
|Previous Message||Bharath Rupireddy||2020-06-22 05:55:54||[PATCH] postgres_fdw connection caching - cause remote sessions linger till the local session exit|