Re: Implementing Incremental View Maintenance

From: Tatsuo Ishii <ishii(at)sraoss(dot)co(dot)jp>
To: thomas(dot)munro(at)gmail(dot)com
Cc: nagata(at)sraoss(dot)co(dot)jp, pgsql-hackers(at)postgresql(dot)org, kgrittn(at)gmail(dot)com
Subject: Re: Implementing Incremental View Maintenance
Date: 2019-07-11 02:39:36
Message-ID: 20190711.113936.1903640014094147259.t-ishii@sraoss.co.jp
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

> I am quite interested to learn how IVM interacts with SERIALIZABLE.

Just for a fun, I have added:
SET TRANSACTION ISOLATION LEVEL SERIALIZABLE;

right after every BEGIN; in incremental_matview.sql in regression test
and it seems it works.

> A couple of superficial review comments:
>
> + const char *aggname = get_func_name(aggref->aggfnoid);
> ...
> + else if (!strcmp(aggname, "sum"))
>
> I guess you need a more robust way to detect the supported aggregates
> than their name, or I guess some way for aggregates themselves to
> specify that they support this and somehow supply the extra logic.
> Perhaps I just waid what Greg Stark already said, except not as well.

I guess we could use moving aggregate (or partial aggregate?)
functions for this purpose, but then we need to run executor directly
rather using SPI. It needs more codes...

> + "WITH t AS ("
> + " SELECT diff.__ivm_count__,
> (diff.__ivm_count__ = mv.__ivm_count__) AS for_dlt, mv.ctid"
> + ", %s"
> + " FROM %s AS mv, %s AS diff WHERE (%s) = (%s)"
> + "), updt AS ("
> + " UPDATE %s AS mv SET __ivm_count__ =
> mv.__ivm_count__ - t.__ivm_count__"
> + ", %s "
> + " FROM t WHERE mv.ctid = t.ctid AND NOT for_dlt"
> + ") DELETE FROM %s AS mv USING t WHERE
> mv.ctid = t.ctid AND for_dlt;",
>
> I fully understand that this is POC code, but I am curious about one
> thing. These queries that are executed by apply_delta() would need to
> be converted to C, or at least used reusable plans, right? Hmm,
> creating and dropping temporary tables every time is a clue that the
> ultimate form of this should be tuplestores and C code, I think,
> right?

Yes, we could reuse the temp tables and plans.

> Then our unofficial automatic CI system[1] will run these tests every
> day, which sometimes finds problems.
>
> [1] cfbot.cputube.org

I appreciate that you provide the system.

Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2019-07-11 02:49:21 Re: Index Skip Scan
Previous Message Thomas Munro 2019-07-11 02:24:17 Re: Copy data to DSA area