Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

From: "Aya Iwata (Fujitsu)" <iwata(dot)aya(at)fujitsu(dot)com>
To: "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [WIP]Vertical Clustered Index (columnar store extension) - take2
Date: 2025-05-17 16:31:30
Message-ID: OS7PR01MB11964A3BD953FF54CF969484BEA92A@OS7PR01MB11964.jpnprd01.prod.outlook.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi hackers,

I will share the notes on the discussions in PGConf.dev. Thanks all for participation.

Feedback on the Advanced Patch Feedback Session;

- A Basic idea that allows both OLTP/OLAP workloads on the same table is OK
- Buffering mechanism has already been used by existing code GIN, IIUC
- IAM seems a more proper approach than TAM
- One reason is that VCI can only optimize the data lookup stuff
- TAM needs all possible queries; it's too much
- Divide the patch more and more
- to allow committers to consider pros and cons and push one by one
- 15 patches is the maximum amount
- Separate features to some committable group

Sawada san suggested several points;
1. Find parts that are useful not only for the VCI and dispatching new threads
2. Develop codes incrementally. E.g., VCI has a custom scan, but we may
be able to give up on the first version

Also, I have questions and advice below on May 16th.

- How to handle visibility?
- What if transactions that insert tuples are aborted?
- VCI vs Index Only Scan
- VCI seems the same as Index Only Scan
- VCI has compression. So, is the amount of size better than the Index?
- How about the efficiency for storage? How much data can we store?
- using TAM (Alvalo's suggestion)
- Most of the code can be ported from the heap. ROS and WOS can exist as forks
of main files, like .vm and .fm. API can be like:

```
CREATE TABLE foo (id, product, price) USING vci WITH (to_be_indexed=price);
```

Again, I really appreciate your efforts.

Regards,
Aya Iwata
Fujitsu Limited

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Sadeq Dousti 2025-05-17 16:38:03 Re: Possible regression in PG18 beta1
Previous Message Sadeq Dousti 2025-05-17 15:45:07 Possible regression in PG18 beta1