Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

Hi Japin,

Thanks for your README questions.

On Fri, Jul 11, 2025 at 7:18 PM Japin Li <japinli(at)hotmail(dot)com> wrote:
...
>
> 3.
> In the README, 'TID' seems to have conflicting definitions:
> Transaction ID (2.1) vs. tuple physical identifier (2.3.1).
>
> Could you confirm the intended meaning? Suggest using 'XID' for Transaction ID
> if my understanding is correct.
>

Yes, TID was meant only for the Tuple identifier. Some terms became
muddled. Hopefully, those are fixed now.

> 4.
> -1: TID relation (maps CRID to original TID)
> -5: TID-CRID mapping table
>
> I'm trying to understand the distinctions here. Based on the definition in
> vci_tidcrid.h, it seems plausible to use just one relation for the mapping,
> suggesting a potential redundancy.
>
> /*
> * TID-CRID pair used for TIDCRID update list
> */
> typedef struct vcis_tidcrid_pair_item
> {
> ItemPointerData page_item_id; /* TID on the original relation */
> vcis_Crid crid; /* CRID */
> } vcis_tidcrid_pair_item_t;
>
> How they are different? I see the code in vci_tidcrid.c
>

AFAIK, the distinction is described by the code comments in vci_columns.h:

+/** Column ID of special column */
+#define VCI_COLUMN_ID_TID (-1)
+#define VCI_COLUMN_ID_NULL (-2)
+#define VCI_COLUMN_ID_DELETE (-3)

So those are all special columns in the ROS data part. In other words,
these internal relations all have data that is indexed by the CRID –
e.g “Delete vector” (2.3.3) and “Null information” (2.3.4). So here,
the TID relation is the mapping from the CRID back to the original
TID.

On the other hand, the other relations...

+/** The data below are not column-stored data.
+ * We prepare them for convenience.
+ */
+#define VCI_COLUMN_ID_TID_CRID (-5)
+#define VCI_COLUMN_ID_TID_CRID_UPDATE (-6)
+#define VCI_COLUMN_ID_TID_CRID_WRITE (-7)
+#define VCI_COLUMN_ID_TID_CRID_CDR (-8)
+#define VCI_COLUMN_ID_DATA_WOS (-9)
+#define VCI_COLUMN_ID_WHITEOUT_WOS (-10)

… are not “column-stored” – In other words, these ones, including the
"TID-CRID mapping table” (-5), are *not* indexed by CRID.

You may be right about a potential redundancy. But right now we're
focused on making these patches ready for open source - removing dead
code to shrink the size, improving the PostgreSQL core interface, and
fixing bugs. Rewriting or optimising the logic will have to wait.

> 5.
> Typo in README.
> - Each extent can have its own independent compression dictionary or all
> extents can share a comon dictionary
> --> s/comon/common/g
>

Fixed.

~~~

Please see the updated README that I attached in the previous post.

======
Kind Regards,
Peter Smith.
Fujitsu Australia