Re: Partitionwise join fails under GEQO

On Wed, Oct 7, 2020 at 12:51 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com> writes:
> > On Mon, Oct 5, 2020 at 11:59 PM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >> ... we could avoid the growth in eclass members for large partition sets
> >> if we simply didn't store child eclass members, instead translating
> >> on-the-fly when we need to deal with child rels. I have a patch
> >> about half done, but it won't be possible to determine the true
> >> performance implications of that idea until it's all done. More
> >> later.

+1 to this idea. We've seen mainly get_eclass_for_sort_expr() become
a bottleneck with large partition sets and getting rid of it would be
really great.

> > If we translate more than ones, every time someone comes searching for
> > an EC member, we will leak memory in the planner memory context, which
> > anyway gets bloated because of the huge number of translations even
> > when done only once. So that needs to be done rather carefully.
>
> I'm not terribly concerned about that. There's not a "huge number"
> of such translations to be done --- it's more like one per index.
> (Also, we could very possibly build the translations in a temp
> context that gets reset every so often, if it becomes an issue.)

Yeah, the memory bloat from this should be minimal. AFAICS, the only
place that demands a child expression from a given matched EC thus
needing a translation is createplan.c, which I would expect to be a
place that doesn't spend a lot of memory because there's no
exploratory processing.

> I am a bit worried about whether we'll be spending a lot more cycles
> to do the added translations; though some of that should be bought
> back by having fewer EC members to compare to. In any event, testing
> a working patch will be a lot more illuminating than speculation.

Agreed.

--
Amit Langote
EDB: http://www.enterprisedb.com

[1] https://www.postgresql.org/message-id/CAApHDvrEXcadNYAAdq6RO0eKZUG6rRHXJGAbpzj8y432gCD9bA%40mail.gmail.com