Re: Foreign join pushdown vs EvalPlanQual

From: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
To: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>, Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, 花田茂 <shigeru(dot)hanada(at)gmail(dot)com>
Subject: Re: Foreign join pushdown vs EvalPlanQual
Date: 2015-08-27 07:52:05
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

> > As I have said repeatedly, it is software design decision by the author
> > of extension. Even if it consumes 100 times larger memory and 1000 times
> > slower, it is his decision and responsibility.
> > Why he has to be forced to use a particular logic despite his intension?
> I don't understand what you proposed,
What I'm talking about is philosophy of software/interface design.
I understand EPQ recheck by alternative plan is "one" reasonable way,
however, people often have different ideas and may be better than
your idea depending on its context/environment/prerequisites/etc...
It is always unpredictable, only God can know what is the best solution.

In other words, I didn't talk about taste of restaurant, the problem is
lack of variation on the menu. You may not want, but we have freedom to
eat terrible taste meal.

> but ISTM that your proposal is
> more like a feature, rather than a bugfix.
Yes, the problem we are facing is lack of a feature. It might be my
oversight when I designed join pushdown infrastructure. Sorry.
So, it is quite natural to add the missing piece to fix up the bug.

> For what you proposed, I
> think we should also improve the existing EPQ mechanism including the
> corresponding FDW routines. One possible improvement is the behavior of
> late row locking. Currently, we do that by 1) re-fetching each
> component tuple from the foreign table after locking it by
> RefetchForeignRow and then 2) if necessary, doing an EPQ recheck, ie,
> re-running the query locally for such component tuples by the core. So,
> if we could re-run the join part of the query remotely without
> tranferring such component tuples from the foreign tables, we would be
> able to not only avoid useless data transfer but improve concurrency
> when the join fails.
> So, how about addressing this issue in two steps; first, work on the
> bugfix patch in [1], and then, work on what you propsed. The latter
> would need more discussion/work, so I think it would be better to take
> that in 9.6. If it's OK, I'll update the patch in [1] and add it to the
> upcoming CF.
It seems to me too invasive for bugfix, and assumes a particular solution.
Please do the rechecking part in the extension, not in the core.

> >> I don't object about your idea either, but I have a concern about that;
> >> it looks like that the more flexiblity we provide, the more the FDWs
> >> implementing their own EPQ would be subject to an internal change in the
> >> core.
> > We never guarantee interface compatibility across major versions. All we
> > can say is 'best efforts'. So, it is always role of extension owner, as
> > long as he continue to maintain his module.
> I think we cannot 100% guarantee the compatibility. That is why I think
> we should avoid an FDW improvement that would be subject to an internal
> change in the core, unless there is a good reason or use-case for that.
It does not make sense unless we don't provide stable and well specified
interface, because developers will have validation and adjustment of their
extension to new major versions.

NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Etsuro Fujita 2015-08-27 08:30:27 Re: Foreign join pushdown vs EvalPlanQual
Previous Message Etsuro Fujita 2015-08-27 07:00:57 Re: Foreign join pushdown vs EvalPlanQual