Re: Foreign join pushdown vs EvalPlanQual

On 2015/08/27 11:08, Kouhei Kaigai wrote:
>> On 2015/08/26 18:01, Kouhei Kaigai wrote:
>>>>> You may think execution of alternative plan is the best way for EPQ rechecks,
>>>>> however, other folks may think their own implementation is the best for EPQ
>>>>> rechecks. I never argue which approach is better.
>>>>> What I point out is freedom/flexibility of implementation choice.

>> Maybe my explanation was not accurate, but I just want to know use
>> cases, to understand the need to provide the flexiblity.

> Let's assume the following situation:

> Someone wants to implement FPGA acceleration feature on top of FDW.

> It enables to run SQL join workloads on FPGA device, but has equivalent
> fallback routines to be executed if FPGA returned an error.
> On EPQ check case, it is quite natural that he wants to re-use this
> fallback routine to validate EPQ tuple. Alternative plan may consume
> additional (at least not zero) memory and other system resource.

Thanks for the answer, but I'm not still convinced. I think the EPQ
testing shown in that use-case would probably not efficient, compared to
the core's.

> As I have said repeatedly, it is software design decision by the author
> of extension. Even if it consumes 100 times larger memory and 1000 times
> slower, it is his decision and responsibility.
> Why he has to be forced to use a particular logic despite his intension?

I don't understand what you proposed, but ISTM that your proposal is
more like a feature, rather than a bugfix. For what you proposed, I
think we should also improve the existing EPQ mechanism including the
corresponding FDW routines. One possible improvement is the behavior of
late row locking. Currently, we do that by 1) re-fetching each
component tuple from the foreign table after locking it by
RefetchForeignRow and then 2) if necessary, doing an EPQ recheck, ie,
re-running the query locally for such component tuples by the core. So,
if we could re-run the join part of the query remotely without
tranferring such component tuples from the foreign tables, we would be
able to not only avoid useless data transfer but improve concurrency
when the join fails.

So, how about addressing this issue in two steps; first, work on the
bugfix patch in [1], and then, work on what you propsed. The latter
would need more discussion/work, so I think it would be better to take
that in 9.6. If it's OK, I'll update the patch in [1] and add it to the
upcoming CF.

>> I don't object about your idea either, but I have a concern about that;
>> it looks like that the more flexiblity we provide, the more the FDWs
>> implementing their own EPQ would be subject to an internal change in the
>> core.

> We never guarantee interface compatibility across major versions. All we
> can say is 'best efforts'. So, it is always role of extension owner, as
> long as he continue to maintain his module.

I think we cannot 100% guarantee the compatibility. That is why I think
we should avoid an FDW improvement that would be subject to an internal
change in the core, unless there is a good reason or use-case for that.

Best regards,
Etsuro Fujita

[1] http://www.postgresql.org/message-id/55CB2D45.7040100@lab.ntt.co.jp