Re: [HACKERS] Runtime Partition Pruning

Hi.

On 2017/12/16 15:05, David Rowley wrote:
> On 13 December 2017 at 00:33, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
>> PFA the updated patch, this can be applied over the v13 patches [1]
>> over commit 487a0c1518af2f3ae2d05b7fd23d636d687f28f3
>
> Hi Beena,
>
> Thanks for posting an updated patch.
>
> I've been looking over this and I think that the use of the
> PartitionDispatch in set_append_subplan_indexes is not correct. What
> we need here is the index of the Append's subnode and that's not what
> RelationGetPartitionDispatchInfo() gives you. Remember that some
> partitions could have been pruned away already during planning.

A somewhat similar concern is being discussed on the "UPDATE partition
key" thread [1]. In that case, ExecInitModifyTable(), when initializing
tuple routing information to handle the "update partition key" case, will
have to deal with the fact that there might be fewer sub-plans in the
ModifyTable node than there are partitions in the partition tree. That
is, source partitions that planner would have determined after pruning,
could be fewer than possible target partitions for rows from the source
partitions to move to, of which the latter consists of *all* partitions.
So, we have to have a mapping from leaf partition indexes as figured out
by RelationGetPartitionDispatchInfo() (indexes that are offsets into a
global array for *all* partitions), to sub-plan indexes which are offsets
into the array for only those partitions that have a sub-plan. Such
mapping is built (per the latest patch on that thread) by
ExecSetupPartitionTupleRouting() in execPartition.c.

We could do something similar here using a similar code structure. Maybe,
add a ExecSetupPartitionRuntimePruning() in execPartition.c (mimicking
ExecSetupPartitionTupleRouting), that accepts AppendState node.
Furthermore, it might be a good idea to have something similar to
ExecFindPartition(), say, ExecGetPartitions(). That is, we have new
functions for run-time pruning that are counterparts to corresponding
functions for tuple routing.

Thanks,
Amit

[1]
https://www.postgresql.org/message-id/c5e1d4ad-d243-52c5-608b-5dbb7183e465%40lab.ntt.co.jp