Re: Optimization for updating foreign tables in Postgres FDW

From: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
To: Thom Brown <thom(at)linux(dot)com>
Cc: Stephen Frost <sfrost(at)snowman(dot)net>, Albe Laurenz <laurenz(dot)albe(at)wien(dot)gv(dot)at>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Shigeru Hanada <shigeru(dot)hanada(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: Optimization for updating foreign tables in Postgres FDW
Date: 2015-11-26 09:00:54
Message-ID: 5656CA46.4030507@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Thom,

Thank you for paying attention to this!

On 2015/11/25 20:36, Thom Brown wrote:
> On 13 May 2015 at 04:10, Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>> On 2015/05/13 0:55, Stephen Frost wrote:
>>> While the EXPLAIN output changed, the structure hasn't really changed
>>> from what was discussed previously and there's not been any real
>>> involvment from the core code in what's happening here.
>>>
>>> Clearly, the documentation around how to use the FDW API hasn't changed
>>> at all and there's been no additions to it for handling bulk work.
>>> Everything here continues to be done inside of postgres_fdw, which
>>> essentially ignores the prescribed "Update/Delete one tuple" interface
>>> for ExecForeignUpdate/ExecForeignDelete.
>>>
>>> I've spent the better part of the past two days trying to reason my way
>>> around that while reviewing this patch and I haven't come out the other
>>> side any happier with this approach than I was back in
>>> 20140911153049(dot)GC16422(at)tamriel(dot)snowman(dot)net(dot)
>>>
>>> There are other things that don't look right to me, such as what's going
>>> on at the bottom of push_update_down(), but I don't think there's much
>>> point going into it until we figure out what the core FDW API here
>>> should look like. It might not be all that far from what we have now,
>>> but I don't think we can just ignore the existing, documented, API.

>> OK, I'll try to introduce the core FDW API for this (and make changes to the
>> core code) to address your previous comments.

> I'm a bit behind in reading up on this, so maybe it's been covered
> since, but is there a discussion of this API on another thread, or a
> newer patch available?

Actually, I'm now working on this. My basic idea is to add new FDW APIs
for handling the bulk work, in order not to make messy the prescribed
"Update/Delete one tuple" interface; 1) add to nodeModifyTable.c or
nodeForeignscan.c the new FDW APIs BeginPushedDownForeignModify,
ExecPushedDownForeignModify and EndPushedDownForeignModify for that, and
2) call these FDW APIs, instead of BeginForeignModify,
ExecForeignUpdate/ExecForeignDelete and EndForeignModify, when doing
update pushdown. I'd like to propose that in more detail as soon as
possible, probably with an updated patch.

Best regards,
Etsuro Fujita

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Magnus Hagander 2015-11-26 09:45:01 Re: pg_stat_replication log positions vs base backups
Previous Message Simon Riggs 2015-11-26 08:45:53 Re: parallel joins, and better parallel explain