Re: [HACKERS] Runtime Partition Pruning

On 21 December 2017 at 23:51, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
> On Thu, Dec 21, 2017 at 2:31 PM, David Rowley
> <david(dot)rowley(at)2ndquadrant(dot)com> wrote:
>> On 19 December 2017 at 21:54, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
>
>> The problem is down to the logic in choose_custom_plan() only choosing
>> a generic plan if the average cost of the generic plan is less than
>> the average custom plan cost. The problem is that the generic plan can
>> have many extra Append subnodes in comparison to the custom plan, all
>> of which are taken into account in the total plan cost, but these may
>> be pruned during execution. The logic in choose_custom_plan() has no
>> idea about this..
>
> I had mentioned this in the first mail on this thread that the generic
> plan is always preferred and Robert said that it is not in scope of
> this patch. Maybe we can start a new thread for this.

Sorry, I missed that. I knew you had modified choose_custom_plan()
too, but I didn't know the diagnosis of the problem had made its way
here yet.

I wonder if Robert understands the extent of the problem. The patch
will be useful to prune away partitions when the partitioned table is
on the inside of a parameterised nested loop join, but I think we'll
likely get some complaints about PREPARE statements always using a
custom plan. It's going to make this pretty hard to write regression
tests for unless we went and invent some GUC "prefer_generic_plan", or
"generic_plan_cost_multiplier" to provide a way to coax
choose_custom_plan() into not choosing a custom plan. I admit to not
having the answer to this but I don't think we should rule out
discussing possible fixes.

> Thank you for working on this. I will look into this and merge with
> my current version of patch and Amit's v16 patches and post a new
> patch soon.

okay. I've just rebased mine on top of Amit's v16 now and will send
shortly. I already took some code from your v7 patch to make my v2
work. Which parts do you think still need to be merged?

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services