Re: [HACKERS] Runtime Partition Pruning

On 1 January 2018 at 19:22, Beena Emerson <memissemerson(at)gmail(dot)com> wrote:
> I think you are testing without asserts

Yeah, I was indeed. Oops.

> The following assert fails: src/backend/optimizer/plan/setrefs.c :
> set_plan_refs: ln 921
> Assert(splan->plan.qual == NIL);
> Append node now has runtime partition quals.
>
> Also since the valid subplans are set in ExecInitAppend, the queries
> with Init Plans do not work. I had moved it to ExecAppend in my patch
> to handle the InitPlans as well.

Thanks for noticing. I've now changed things around so this case works
as it should and I've added a test too.

I've attached an updated patch which also fixes a number of other
problems with my previous patch.

1. The Bitmapset I was using in nodeAppend.c to mark the valid
subplans was pretty bogus for Parallel Append since the memory for the
set was not in shared memory. I changed things around to reuse the
pa_finished[] array and the patch just now sets pa_finished to true
for any invalid subplans.
2. I've added a new memory context to use in nodeAppend.c which is
used to call the planner code to determine which partitions are valid.
I'd been trying to have Amit be careful to pfree() everything in his
v17 patch, but I realised it was just not possible to get everything
pfree'd. I found it pretty easy to construct a test case which caused
an OOM.
3. I've added support for IN lists to be pruned when the IN() list
contains a parameter. The changes I made to support this case probably
mostly belong in Amit's faster partition pruning patch, but I've put
them here for now to get this case working. There's a bunch of new
tests to test this.
4. Various other cosmetic improvements.

The attached patch should be applied after patching master with Amit's
v17 faster partition pruning patch [1].

[1] https://www.postgresql.org/message-id/58c3e20a-a964-4fdb-4e7d-bd833e9bead1@lab.ntt.co.jp

--
David Rowley http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services