Re: Parallel append plan instability/randomness

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> [ example where current parallel-append dispatch algorithm loses ]

I should clarify my thinking a bit more: I was imagining that we might
switch to a system where, as each worker comes free, it makes a dynamic
choice of which subplan to take next. That means that not only do we have
more knowledge of the number of available workers than the planner did,
but we also have hard knowledge of which subplans actually finished first.
So even (or especially?) granted that the cost estimates are imperfect,
that might provide a leg up in choosing how to schedule the remaining
tasks.

I haven't thought about it hard enough to have a clear idea of how that
might win. But for sure I don't think we should dismiss out of hand
the idea that it could be better than choosing a fixed dispatch order
at plan time.

Anyway, I'm merely suggesting that as an approach worth investigating in
future research; I doubt anyone is going to produce an implementation
right away. So we still want to figure out exactly what happened on
silverfish and how we can nail down that expected plan shape. (Or,
perhaps, decide that we don't need to see that plan in the output?)

regards, tom lane