Re: [HACKERS] Runtime Partition Pruning

(sending my reply in parts for concurrency)

On 6 April 2018 at 14:39, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp> wrote:
> I think we can save some space here by not having the pointers stored
> here. Instead of passing the pointer itself in the recursive calls to
> find_subplans_for_extparams_recurse, et al, just pass the entire array and
> an offset to use for the given recursive invocation.

hmm, where would those offsets be stored? I don't want to have to do
any linear searching to determine the offset, which is why I just
stored the pointer to the flat array. It seems very efficient to me to
do this. Remember that this pruning can occur per tuple in cases like
parameterized nested loops.

Are you worried about memory consumption? It's one pointer per
partition. I imagine there's lots more allocated for DML on a
partitioned table as it needs to store maps to map attribute numbers.

Or are you thinking the saving of storing an array of 32-bit int
values is better than the array of probably 64-bit pointers? So
requires half the space?

> If you look at ExecFindPartition used for tuple routing, you may see that
> there no recursion at all. Similarly find_subplans_for_extparams_recurse,
> et al might be able to avoid recursion if similar tricks are used.

That seems pretty different. That's looking for a single node in a
tree, so just is following a single path from the root, it never has
to go back up a level and look down any other paths.

What we need for the find_subplans_for_extparams_recurse is to find
all nodes in the entire tree which match the given clause. Doing this
without recursion would require some sort of stack so we can go back
up a level and search again down another branch. There are ways
around this without using recursion, sure, but I don't think any of
them will be quite as convenient and simple. The best I can think of
is to palloc some stack manually and use some depth_level to track
which element to use. An actual stack seems more simple. I can't
quite think of a good way to know in advance how many elements we'd
need to palloc.

> Finally about having two different functions for different sets of Params:
> can't we have just named find_subplans_for_params_recurse() and use the
> appropriate one based on the value of some parameter? I can't help but
> notice the duplication of code.

I had decided not to make this one function previously as I didn't
really want to add unnecessary branching in the code. After
implementing it, it does not look as bad as I thought.

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services