Re: nbtree: opt-in leaf key prefix compression

From: Greg Burd <greg(at)burd(dot)me>
To: Yao Feng <fengyao0087(at)gmail(dot)com>
Cc: pgsql-hackers(at)lists(dot)postgresql(dot)org, Peter Geoghegan <pg(at)bowt(dot)ie>
Subject: Re: nbtree: opt-in leaf key prefix compression
Date: 2026-11-18 17:31:49
Message-ID: 179502032980.1724456.1423341864991075612@burd.me
Views: Whole Thread | Raw Message | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hello,

v13 attached, rebased onto 45047c51aab.

Further testing found that fitting on a page does not guarantee that an
encoded tuple or prefix directory fits its size field. I added
admission checks for those limits, plus checks for final build leaves
whose last item is a posting list. The new tests reproduce both cases
on 32KB builds.

I also found why some incrementally built indexes grew larger with
compression enabled. The coded-page split picker balanced the halves
even on rightmost pages, ignoring fillfactor. Some pages that could
be decoded onto one page also unnecessarily bypassed the ordinary
split picker. The patch now uses that picker when decoding fits and
honors rightmost fillfactor among legal split candidates otherwise.

For the fixed-width path workload, that reduced the 100,000-row index
from 8,118,272 to 5,414,912 bytes; compression off used 5,562,368 bytes.
The attached notes give the other results and reproduction details.
These are size observations, not timing results. Variable-length
headers still limit sharing, and the change does not shrink every
workload or fillfactor setting.

Other changes since v12:

0001: Validate coded metadata before decoding, including raw pageinspect
input, and add corruption tests. Copy amcheck's saved heap TID so the
next decode cannot overwrite a duplicate-key diagnostic. Extend unique
index, NULL, representation-preservation, and standby coverage.

This remains a prototype, but a highly functional one. The fixes do not
settle the format choice or establish that scan CPU costs and
default-off overhead are acceptable.

best.

-greg

Attachment Content-Type Size
notes.txt text/plain 3.0 KB
v13-0001-nbtree-add-opt-in-leaf-key-prefix-compression.patch application/octet-stream 225.5 KB

In response to

Browse pgsql-hackers by date

  From Date Subject
Next Message Bharath Rupireddy 2026-11-18 17:42:08 Re: [PATCH] Fall back to serializing when parallel apply runs out of DSM segments
Previous Message Bharath Rupireddy 2026-11-18 17:29:49 Re: Parallel vacuum: check XID wraparound failsafe between indexes