Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.

From: "Jonathan S(dot) Katz" <jkatz(at)postgresql(dot)org>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andres Freund <andres(at)anarazel(dot)de>, Ashutosh Bapat <ashutosh(dot)bapat(at)enterprisedb(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>, Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>
Subject: Re: Expression errors with "FOR UPDATE" and postgres_fdw with partition wise join enabled.
Date: 2018-08-14 21:06:31
Message-ID: DD0D64DA-2B57-4C85-8CDD-D5BE62E0F0EA@postgresql.org
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers


> On Aug 14, 2018, at 11:51 AM, Robert Haas <robertmhaas(at)gmail(dot)com> wrote:
>
> On Mon, Aug 13, 2018 at 12:32 PM, Etsuro Fujita
> <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp> wrote:
>> One thing I noticed might be an improvement is to skip
>> build_joinrel_partition_info if the given joinrel will be to have
>> consider_partitionwise_join=false; in the previous patch, that function
>> created the joinrel's partition info such as part_scheme and part_rels if
>> the joinrel is considered as partitioned, independently of the flag
>> consider_partitionwise_join for it, but if that flag is false, we don't
>> generate PWJ paths for the joinrel, so we would not need to create that
>> partition info at all. This would not only avoid unnecessary processing in
>> that function, but also make unnecessary the changes I made to
>> try_partitionwise_join, generate_partitionwise_join_paths,
>> apply_scanjoin_target_to_paths, and create_ordinary_grouping_paths. So I
>> updated the patch that way. Please find attached an updated version of the
>> patch.
>
> I guess the question is whether there are (or might be in the future)
> other dependencies on part_scheme. For example, it looks like
> partition pruning uses it. I'm not sure whether partition pruning
> supports a plan like:
>
> Append
> -> Nested Loop
> -> Seq Scan on p1
> -> Index Scan on q1
> <repeat the above for p2/q2 etc.>
>
> If it doesn't, that's just an implementation restriction; somebody
> might want to fix things so it works, if it doesn't already.

While we (the RMT) are happy to see there has been progress on this thread,
11 Beta 3 was released containing this problem and the time to adequately
test this feature prior to GA release is rapidly dwindling.

The RMT would like to see this prioritized and resolved. At this point we have
considered the option of having partitionwise joins completely disabled in
PostgreSQL 11. This is preferably not the path we want to choose, but this
option is now on the table and we will issue an ultimatum soon if we do not see
further progress.

In the previous discussion, the generally accepted solution for PostgreSQL 11
seemed to be to disable the problematic case and any further work would be for
PostgreSQL 12 only. If something is different, please indicate why ASAP and
work to resolve. The RMT will also conduct some additional technical analysis
as well.

Sincerely,

Alvaro, Andres, Jonathan

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Todd A. Cook 2018-08-14 21:07:28 Re: buildfarm: could not read block 3 in file "base/16384/2662": read only 0 of 8192 bytes
Previous Message Fabien COELHO 2018-08-14 20:53:48 Re: libpq should not look up all host addresses at once