Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

On Fri, Jul 11, 2025 at 1:46 PM shveta malik <shveta(dot)malik(at)gmail(dot)com> wrote:
>
> Thank You for working on this. I started going through the README and
> tried running simple tests, have few concerns:
>
> 1)
> I am not able to understand section 4.2 'WOS-to-ROS conversion'. When
> whiteout-WOS says 'delete 4', what does that mean? 4 is CRID, TXID?
> And when does delete-vector X represents? I did not get why ColA-2,
> ColA-4 and ColB-2, ColB-4 were removed in resultant data? Is the
> diagram complete?
>
> 2)
> We can make the definition consistent at both places as the first one
> gives a feeling that rows are marked for deletion in WOS while the
> second one says ROS.
>
> Whiteout WOS = Record of WOS rows marked for deletion
> Whiteout WOS -- TID records of WOS rows that are marked for deletion on ROS
>
> 3)
> It is not part of README. But please help me understand the meaning
> and usage of this GUC in VCI context:
> vci.max_devices: Sets the maximum device number which can be attached.
>
> 4)
> I was trying my hands on using VCI, I found that on enabling the VCI
> path, autovacuum asserts in between. We can reproduce the scenario
> using:
> create extension vci;
> create table tab1( i int, j int, k int);
> create index ivci on tab1 using vci (i);
> insert into tab1 SELECT generate_series(1,1000),
> generate_series(1,1000), generate_series(1,1000);
> insert into tab1 SELECT generate_series(1,1000),
> generate_series(1,1000), generate_series(1,1000);
> SET enable_seqscan TO off;
> VACUUM ANALYZE public.tab1;
>
> It asserts here:
> TRAP: failed Assert("TransactionIdIsNormal(xmax)"), File: "heapam.c",
> Line: 7373, PID: 79834
> postgres: autovacuum worker postgres(ExceptionalCondition+0xbb)[0x55fdf8f3ccc6]
> postgres: autovacuum worker
> postgres(heap_pre_freeze_checks+0x18f)[0x55fdf877e26f]
> postgres: autovacuum worker postgres(index_vacuum_cleanup+0x181)[0x55fdf87a1c06]
> postgres: autovacuum worker postgres(vac_cleanup_one_index+0x27)[0x55fdf8a0503a]
>
> -------------
>
> Few typos in README:
> a) Each VCI indexed column is stored as an internal relations.
> --relations --> relation
>
> b) Records are addresses by CRID (Columnar Record ID) instead of by TID.
> --addresses->addressed
>
> c) Extents can be found by ID using offsets in a column "meta-data"
> internal relation.
> -- by ID using offsets? Do you mean 'by using offsets' alone?
>
> d) EXPLAIN ANALYSE -->EXPLAIN ANALYZE
>
>

Hi Shveta.

Thanks for your README questions and typo reports. I will try to
address those soon.

Regarding your reported VACUUM exception, I believe this is the same
issue that was previously reported by Japin Li [1].

======
[1] https://www.postgresql.org/message-id/SY8P300MB0442BEC3F5CF432F0121ACC4B642A%40SY8P300MB0442.AUSP300.PROD.OUTLOOK.COM

Kind Regards,
Peter Smith.
Fujitsu Australia