Re: [WIP]Vertical Clustered Index (columnar store extension) - take2

From: Peter Smith <smithpb2250(at)gmail(dot)com>
To: Tomas Vondra <tomas(at)vondra(dot)me>
Cc: "Aya Iwata (Fujitsu)" <iwata(dot)aya(at)fujitsu(dot)com>, "pgsql-hackers(at)lists(dot)postgresql(dot)org" <pgsql-hackers(at)lists(dot)postgresql(dot)org>
Subject: Re: [WIP]Vertical Clustered Index (columnar store extension) - take2
Date: 2025-05-29 08:08:20
Message-ID: CAHut+Pvmtzx5vL40Js6enQwtakfBnqNCrjo964+-5oe27Xbi=g@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi Tomas.

> What I really miss in the current patch is some sort of READMEs with
>
> - high-level design of the VCI indexes
>
> - description of the consistency/visibility model (does it behave the
> same way as querying the heap, can it be out of sync for, ...)
>
> - WOS/ROS architecture (when are rows inserted into WOS, promoted into
> ROS, what triggers that, ...)
>
> - what's the in-memory / on-disk format
>
> - some places in the patch mention "internal VCI tables" but I have no
> idea what that is
>
> - how does the execution work? compression? crucial points to consider
> for optimal performance, etc.
>
> - limitations (temporary - can be relaxed in the future, permanent -
> inherent to the columnar design) and trade-offs
>
> - what are the various custom scan executor nodes
>
> - what "background" processes happen (custom workers, ...)
>
> - anything else substantial for understanding the design
>
> Maybe there's some of this in the 0002 patch, but I haven't stumbled
> over it so far.

Acknowledged. We are working now on creating a draft README and will
post it here ASAP.

~~~

For patch 0001 you reported some items as being "unused". You were
correct on all counts; those are all removed in the latest patch set.

> gen_node_support.c
> ------------------
> - What's SmcAllocSetContext for? Doesn't seem to be mentioned anywhere.

Removed.

> params.c
> ------------------
> - ExecInitParam_hook seems unused, no?

Removed.

> lwlock.c
> ------------------
> - Vci_lwlock_kind seems unused.

Removed.

> memnodes.h
> ------------------
> - What's SmcAllocSetContext? It's not defined anywhere.

Removed.

> params.h
> ------------------
> - Adds noNeedLoadFromMain, loadedFromMain to ParamExecData, but it
> doesn't seem to be used anywhere (in either patch).

Removed.

> lwlock.h
> ------------------
> - LWLockKind / Vci_lwlock_kind seems unused

Removed

======
Kind Regards,
Peter Smith.
Fujitsu Australia.

Attachment Content-Type Size
v4-0003-VCI-documentation.patch application/octet-stream 6.3 KB
v4-0001-Add-postgres-code-fix-needed-for-VCI.patch application/octet-stream 79.3 KB
v4-0002-Add-VCI-module.patch application/octet-stream 1.3 MB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message m.litsarev 2025-05-29 08:23:47 Warning -Wclobbered in PG_TRY(...)
Previous Message Rahila Syed 2025-05-29 07:47:21 Re: Persist injection points across server restarts