Skip partition tuple routing with constant partition key

From: "houzj(dot)fnst(at)fujitsu(dot)com" <houzj(dot)fnst(at)fujitsu(dot)com>
To: PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Skip partition tuple routing with constant partition key
Date: 2021-05-17 11:36:48
Message-ID: OS0PR01MB571649B27E912EA6CC4EEF03942D9@OS0PR01MB5716.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

When loading some data into a partitioned table for testing purpose,

I found even if I specified constant value for the partition key[1], it still do

the tuple routing for each row.

[1]---------------------

UPDATE partitioned set part_key = 2 , …

INSERT into partitioned(part_key, ...) select 1, …

---------------------

I saw such SQLs automatically generated by some programs,

So , personally, It’d be better to skip the tuple routing for this case.

IMO, we can use the following steps to skip the tuple routing:

1) collect the column that has constant value in the targetList.

2) compare the constant column with the columns used in partition key.

3) if all the columns used in key are constant then we cache the routed partition

 and do not do the tuple routing again.

In this approach, I did some simple and basic performance tests:

----For plain single column partition key.(partition by range(col)/list(a)...)

When loading 100000000 rows into the table, I can see about 5-7% performance gain

for both cross-partition UPDATE and INSERT if specified constant for the partition key.

----For more complicated expression partition key(partition by range(UDF_func(col)+x)…)

When loading 100000000 rows into the table, it will bring more performance gain.

About > 20% performance gain

Besides, I did not see noticeable performance degradation for other cases(small data set).

Attaching a POC patch about this improvement.

Thoughts ?

Best regards,

houzj

Attachment Content-Type Size
0001-skip-tuple-routing-for-constant-partition-key.patch application/octet-stream 11.6 KB

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message vignesh C 2021-05-17 12:18:45 Re: subscriptioncheck failure
Previous Message Ranier Vilela 2021-05-17 11:28:46 Re: Possible memory corruption (src/timezone/zic.c b/src/timezone/zic.c)