Re: Incremental View Maintenance, take 2

Hi, all

On Aug 30, 2025 at 04:04 +0800, Yugo Nagata <nagata(at)sraoss(dot)co(dot)jp>, wrote:
>
> Anyway, I will start rebasing the patches, reorganizing the patch set,
> and applying fixes made in pg_ivm [2].

First of all, we would like to express our sincere gratitude for your continuous efforts and contributions to the IVM feature.
Our AQUMV (Answer Query Using Materialized Views) functionality in Apache Cloudberry is built directly upon your foundational work.

We recently encountered an issue in one of our customer's production environments.
The problem occurs in the function CreateIvmTriggersOnBaseTablesRecurse(), where a Bitmapset relidsis used to record rte->relid.

```
case T_RangeTblRef:
{
 int rti = ((RangeTblRef *) node)->rtindex;
 RangeTblEntry *rte = rt_fetch(rti, qry->rtable);

 if (rte->rtekind == RTE_RELATION && !bms_is_member(rte->relid, *relids))
 {
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, TRIGGER_TYPE_BEFORE, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, TRIGGER_TYPE_BEFORE, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, TRIGGER_TYPE_BEFORE, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, TRIGGER_TYPE_BEFORE, true);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_INSERT, TRIGGER_TYPE_AFTER, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_DELETE, TRIGGER_TYPE_AFTER, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_UPDATE, TRIGGER_TYPE_AFTER, ex_lock);
 CreateIvmTrigger(rte->relid, matviewOid, TRIGGER_TYPE_TRUNCATE, TRIGGER_TYPE_AFTER, true);

 *relids = bms_add_member(*relids, rte->relid);
 }
}
```
However,  the Bitmapset structure is typically designed to store small integers (such as relation index), whereas rte->relidrepresents the Oid of a relation.
For instance, when a table is created immediately after initializing a new cluster, its Oid might be 17019. Storing such a value in a Bitmapset consumes approximately 0.2MB of memory when creating an IVM.

(gdb) p bmsToString(((Bitmapset *) relids))
$13 = 0x5643d7bbb070 "(b 17019)"
(gdb) p *((Bitmapset *) relids)
$14 = {nwords = 266, words = 0x5643d79d1bd8}

This memory usage becomes even more significant when dealing with larger Oids.
Moreover, a more critical issue arises when the Oid exceeds 0x7FFFFFFF. Since Oid is an unsigned integer, but the Bitmapset expects signed integers, an assertion failure will occur in such cases.

We have encountered similar requirements in our own scenarios and eventually switched to using a HTAB implementation. Hope this proves useful for your work.

--
Zhang Mingli
HashData