Re: Implementing Incremental View Maintenance

From: Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp>
To: nagata(at)sraoss(dot)co(dot)jp
Cc: pgsql-hackers(at)postgresql(dot)org, hoshiai(at)sraoss(dot)co(dot)jp, ishii(at)sraoss(dot)co(dot)jp, michael(at)paquier(dot)xyz, amitlangote09(at)gmail(dot)com, alvherre(at)2ndquadrant(dot)com, thomas(dot)munro(at)gmail(dot)com, kgrittn(at)gmail(dot)com
Subject: Re: Implementing Incremental View Maintenance
Date: 2020-05-08 01:13:06
Message-ID: 20200508.101306.2265966612787422911.t-ishii@sraoss.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

>> Hi,
>>
>> Attached is the latest patch (v15) to add support for Incremental Materialized
>> View Maintenance (IVM). It is possible to apply to current latest master branch.

I have tried to use IVM against TPC-DS (http://www.tpc.org/tpcds/)
queries. TPC-DS models decision support systems and those queries are
modestly complex. So I thought applying IVM to those queries could
show how IVM covers real world queries.

Since IVM does not support queries including ORDER BY and LIMIT, I
removed them from the queries before the test.

Here are some facts so far learned in this attempt.

- Number of TPC-DS query files is 99.
- IVM was successfully applied to 20 queries.
- 33 queries failed because they use WITH clause (CTE) (currenly IVM does not support CTE).
- Error messages from failed queries (except those using WITH) are below:
(the number indicates how many queries failed by the same reason)

11 aggregate functions in nested query are not supported on incrementally maintainable materialized view
8 window functions are not supported on incrementally maintainable materialized view
7 UNION/INTERSECT/EXCEPT statements are not supported on incrementally maintainable materialized view
5 WHERE clause only support subquery with EXISTS clause
3 GROUPING SETS, ROLLUP, or CUBE clauses is not supported on incrementally maintainable materialized view
3 aggregate function and EXISTS condition are not supported at the same time
2 GROUP BY expression not appeared in select list is not supported on incrementally maintainable materialized view
2 aggregate function with DISTINCT arguments is not supported on incrementally maintainable materialized view
2 aggregate is not supported with outer join
1 aggregate function stddev_samp(integer) is not supported on incrementally maintainable materialized view
1 HAVING clause is not supported on incrementally maintainable materialized view
1 subquery is not supported with outer join
1 column "avg" specified more than once

Attached are the queries IVM are successfully applied.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

Attachment Content-Type Size
IVM_sucessfull_queries.tar.gz application/octet-stream 3.9 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Fujii Masao 2020-05-08 01:42:18 Re: Why are wait events not reported even though it reads/writes a timeline history file?
Previous Message Kyotaro Horiguchi 2020-05-08 01:00:09 Re: +(pg_lsn, int8) and -(pg_lsn, int8) operators