Re: Foreign join pushdown vs EvalPlanQual

From: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>
To: Kouhei Kaigai <kaigai(at)ak(dot)jp(dot)nec(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>, 花田茂 <shigeru(dot)hanada(at)gmail(dot)com>
Subject: Re: Foreign join pushdown vs EvalPlanQual
Date: 2015-07-07 10:15:51
Message-ID: 559BA6D7.605@lab.ntt.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 2015/07/06 9:42, Kouhei Kaigai wrote:
>>> Also, I don't want to stick on the assumption that relations involved in
>>> remote join are all managed by same foreign-server no longer.
>>> The following two ideas introduce possible enhancement of remote join
>>> feature that involved local relations; replicated table or transformed
>>> to VALUES() clause.

>>> I think add_paths_to_joinrel() is the best location for foreign-join,
>>> not only custom-join. Relocation to standard_join_search() has larger
>>> disadvantage than its advantage.

>> I agree with you that it's important to ensure the expandability, and my
>> question is, is it possible that the API call from standard_join_search
>> also realize those idea if FDWs can get the join information through the
>> root or something like that?

> I don't deny its possibility, even though I once gave up to implement to
> reproduce join information - which relations and other ones are joined in
> this level - using PlannerInfo and RelOptInfo.

OK

> However, we need to pay attention on advantages towards the alternatives.
> Hooks on add_paths_to_joinrel() enables to implement identical stuff, with
> less complicated logic to reproduce left / right relations from RelOptInfo
> of the joinrel. (Note that RelOptInfo->fdw_private enables to avoid path-
> construction multiple times.)
> I'm uncertain why this API change is necessary to fix up the problem
> around EvalPlanQual.

Yeah, maybe we wouldn't need any API change. I think we would be able
to fix this by complicating add_path as you pointed out upthread. I'm
not sure that complicating it is a good idea, though. I think that it
might be possible that the callback in standard_join_search would allow
us to fix this without complicating the core path-cost-comparison stuff
such as add_path. I noticed that what I proposed upthread doesn't work
properly, though.

Actually, I have another concern about the callback location that you
proposed; that might meaninglessly increase planning time in the
postgres_fdw case when using remote estimates, which the proposed
postgres_fdw patch doesn't support currently IIUC, but I think it should
support that. Let me explain about that. If you have A JOIN B JOIN C
all on the same foreign server, for example, we'll have only to perform
a remote EXPLAIN for A-B-C for the estimates (when adopting a strategy
that we push down a join as large as possible into the remote server).
However, ISTM that the callback in add_paths_to_joinrel would perform
remote EXPLAINs not only for A-B-C but for A-B, A-C and B-C according to
the dynamic programming algorithm. (Duplicated remote EXPLAINs for
A-B-C can be eliminated using a way you proposed.) Thus the remote
EXPLAINs for A-B, A-C and B-C seem to me meaningless while incurring
performance degradation in query planning. Maybe I'm missing something,
though.

Best regards,
Etsuro Fujita

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Kyotaro HORIGUCHI 2015-07-07 10:20:19 Re: RFC: replace pg_stat_activity.waiting with something more descriptive
Previous Message Amit Kapila 2015-07-07 09:46:17 Re: Freeze avoidance of very large table.