Re: Optimization for updating foreign tables in Postgres FDW

From: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>
To: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
Cc: Robert Haas <robertmhaas(at)gmail(dot)com>, Rushabh Lathia <rushabh(dot)lathia(at)gmail(dot)com>, Noah Misch <noah(at)leadboat(dot)com>, Thom Brown <thom(at)linux(dot)com>, Stephen Frost <sfrost(at)snowman(dot)net>, Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Optimization for updating foreign tables in Postgres FDW
Date: 2016-04-15 05:31:51
Message-ID: CAB7nPqQ5RBHo59Mbb3SCOqsnVG-p5nRuUtuXkr6R_=4Kv9kLhg@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Thu, Apr 14, 2016 at 10:44 AM, Etsuro Fujita
<fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> On 2016/04/13 21:50, Michael Paquier wrote:
>> On Wed, Apr 13, 2016 at 9:46 PM, Robert Haas <robertmhaas(at)gmail(dot)com>
>> wrote:
>>> On Tue, Apr 12, 2016 at 10:24 PM, Etsuro Fujita
>>> <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>>>>>
>>>>> How about we encapsulate the while (PQisBusy(...)) loop into a new
>>>>> function pgfdw_get_result(), which can be called after first calling
>>>>> PQsendQueryParams()? So then this code will say dmstate->result =
>>>>> pgfdw_get_result(dmstate->conn). And we can do something similar for
>>>>> the other call to PQexecParams() in create_cursor(). Then let's also
>>>>> add something like pgfdw_exec_query() which calls PQsendQuery() and
>>>>> then pgfdw_get_result, and use that to replace all of the existing
>>>>> calls to PQexec().
>>>>>
>>>>> Then all the SQL postgres_fdw executes would be interruptible, not
>>>>> just the new stuff.
>
> I would be happy if you work on that.

OK, so I have finished with the attached. One thing that I noticed in
the previous patch version is that it completely ignored cases where
multiple PGresult could be returned by server, say when multiple
queries are sent in the same string: PQexec gets always the last one,
so I think that we had better do the same here. I have switched all
the PQexec calls to a custom routine that combines
PQsendQuery/PQgetResult, and PQexecParams is switched to
PQsendQueryParams/PQgetResult. This structure allows all queries run
though postgres_fdw.c to be interruptible.
--
Michael

Attachment Content-Type Size
pgfdw-interrupt-v2.patch text/x-patch 8.8 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Michael Paquier 2016-04-15 05:45:33 Rework help interface of vcregress.pl
Previous Message Fujii Masao 2016-04-15 05:03:08 Re: [COMMITTERS] pgsql: Add regression tests for multiple synchronous standbys.