Re: Foreign join pushdown vs EvalPlanQual

> > As I have said repeatedly, it is software design decision by the author
> > of extension. Even if it consumes 100 times larger memory and 1000 times
> > slower, it is his decision and responsibility.
> > Why he has to be forced to use a particular logic despite his intension?
>
> I don't understand what you proposed,
>
What I'm talking about is philosophy of software/interface design.
I understand EPQ recheck by alternative plan is "one" reasonable way,
however, people often have different ideas and may be better than
your idea depending on its context/environment/prerequisites/etc...
It is always unpredictable, only God can know what is the best solution.

In other words, I didn't talk about taste of restaurant, the problem is
lack of variation on the menu. You may not want, but we have freedom to
eat terrible taste meal.

> but ISTM that your proposal is
> more like a feature, rather than a bugfix.
>
Yes, the problem we are facing is lack of a feature. It might be my
oversight when I designed join pushdown infrastructure. Sorry.
So, it is quite natural to add the missing piece to fix up the bug.

> For what you proposed, I
> think we should also improve the existing EPQ mechanism including the
> corresponding FDW routines. One possible improvement is the behavior of
> late row locking. Currently, we do that by 1) re-fetching each
> component tuple from the foreign table after locking it by
> RefetchForeignRow and then 2) if necessary, doing an EPQ recheck, ie,
> re-running the query locally for such component tuples by the core. So,
> if we could re-run the join part of the query remotely without
> tranferring such component tuples from the foreign tables, we would be
> able to not only avoid useless data transfer but improve concurrency
> when the join fails.
>
> So, how about addressing this issue in two steps; first, work on the
> bugfix patch in [1], and then, work on what you propsed. The latter
> would need more discussion/work, so I think it would be better to take
> that in 9.6. If it's OK, I'll update the patch in [1] and add it to the
> upcoming CF.
>
It seems to me too invasive for bugfix, and assumes a particular solution.
Please do the rechecking part in the extension, not in the core.

> >> I don't object about your idea either, but I have a concern about that;
> >> it looks like that the more flexiblity we provide, the more the FDWs
> >> implementing their own EPQ would be subject to an internal change in the
> >> core.
>
> > We never guarantee interface compatibility across major versions. All we
> > can say is 'best efforts'. So, it is always role of extension owner, as
> > long as he continue to maintain his module.
>
> I think we cannot 100% guarantee the compatibility. That is why I think
> we should avoid an FDW improvement that would be subject to an internal
> change in the core, unless there is a good reason or use-case for that.
>
It does not make sense unless we don't provide stable and well specified
interface, because developers will have validation and adjustment of their
extension to new major versions.

Thanks,
--
NEC Business Creation Division / PG-Strom Project
KaiGai Kohei <kaigai(at)ak(dot)jp(dot)nec(dot)com>