Re: [HACKERS] advanced partition matching algorithm for partition-wise join

From: Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com>
To: amul sul <sulamul(at)gmail(dot)com>
Cc: Thomas Munro <thomas(dot)munro(at)gmail(dot)com>, Robert Haas <robertmhaas(at)gmail(dot)com>, Amit Langote <Langote_Amit_f8(at)lab(dot)ntt(dot)co(dot)jp>, Ashutosh Bapat <ashutosh(dot)bapat(dot)oss(at)gmail(dot)com>, Rajkumar Raghuwanshi <rajkumar(dot)raghuwanshi(at)enterprisedb(dot)com>, Etsuro Fujita <fujita(dot)etsuro(at)lab(dot)ntt(dot)co(dot)jp>, Dmitry Dolgov <9erthalion6(at)gmail(dot)com>, Thomas Munro <thomas(dot)munro(at)enterprisedb(dot)com>, Antonin Houska <ah(at)cybertec(dot)at>, PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: [HACKERS] advanced partition matching algorithm for partition-wise join
Date: 2019-07-08 11:33:32
Message-ID: CAPmGK177W+jNgpM5f_m-vdDKbEBu_=3CyPzFjkT_1nzf1Vqn+A@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On Wed, Jul 3, 2019 at 3:44 PM Etsuro Fujita <etsuro(dot)fujita(at)gmail(dot)com> wrote:
> On Tue, Jul 2, 2019 at 1:47 PM amul sul <sulamul(at)gmail(dot)com> wrote:
> > Attached version is rebase atop of the latest master head(c74d49d41c), thanks.
>
> Thanks! Will review.

I started reviewing this. Here is my initial review comments:

* 0001-Hash-partition-bound-equality-refactoring-v22.patch

First of all, I agree with your view on hash partitioning:

"3. For hash partitioned tables however, we support partition-wise join
only when the bounds exactly match. For hash partitioning it's unusual
to have missing partitions and hence generic partition matching is not
required."

which is cited from the commit message for the main patch
"0002-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi-v22.patch".
(I think it would be better if we can extend the partition matching to
the hash-partitioning case where there are missing partitions in
future, though.) However, I don't think it's a good idea to do this
refactoring, because that would lead to duplicating the code to check
whether two given hash bound collections are equal in
partition_bounds_equal() and partition_hash_bounds_merge() that will
be added by the main patch, after all. To avoid that, how about
calling partition_bounds_equal() from partition_hash_bounds_merge() in
the main patch, like the attached? I also did some cleanup for
partition_hash_bounds_merge(). This change makes the refactoring
patch unnecessary, so I dropped it. Also, I included the
regression-test patch
"0003-Tests-for-0-1-1-1-and-1-0-partition-matching-v22.patch" in the
attached, because make check didn't succeed without the
regression-test patch.

That's it for now. I'll review the remaining parts (ie,
"0002-Partition-wise-join-for-1-1-1-0-0-1-partition-matchi-v22.patch"
and "0003-Tests-for-0-1-1-1-and-1-0-partition-matching-v22.patch")
closely next.

Best regards,
Etsuro Fujita

Attachment Content-Type Size
Improve-partition-matching-for-partitionwise-joins-v23.patch application/octet-stream 283.7 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2019-07-08 11:56:08 Re: Commitfest 2019-07, the first of five* for PostgreSQL 13
Previous Message Peter Eisentraut 2019-07-08 11:07:52 Re: Switching PL/Python to Python 3 by default in PostgreSQL 12