Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

On Sat, May 24, 2025 at 7:29 AM Tomas Vondra <tomas(at)vondra(dot)me> wrote:
>
...
>
> What I really miss in the current patch is some sort of READMEs with
>
> - high-level design of the VCI indexes
>
> - description of the consistency/visibility model (does it behave the
> same way as querying the heap, can it be out of sync for, ...)
>
> - WOS/ROS architecture (when are rows inserted into WOS, promoted into
> ROS, what triggers that, ...)
>
> - what's the in-memory / on-disk format
>
> - some places in the patch mention "internal VCI tables" but I have no
> idea what that is
>
> - how does the execution work? compression? crucial points to consider
> for optimal performance, etc.
>
> - limitations (temporary - can be relaxed in the future, permanent -
> inherent to the columnar design) and trade-offs
>
> - what are the various custom scan executor nodes
>
> - what "background" processes happen (custom workers, ...)
>
> - anything else substantial for understanding the design
>
> Maybe there's some of this in the 0002 patch, but I haven't stumbled
> over it so far.
>

Hi Tomas.

Attached is the first version of the README, intended to address the
points above.

For convenience, I’ve included it as a separate file, but the plan is
to integrate it into the 0002 patch in the next update.

Please let me know if you have any feedback or suggestions for
improving the content.

======
Kind Regards,
Peter Smith.
Fujitsu Australia.