| From: | Greg Burd <greg(at)burd(dot)me> |
|---|---|
| To: | Yao Feng <fengyao0087(at)gmail(dot)com> |
| Cc: | pgsql-hackers(at)lists(dot)postgresql(dot)org, Peter Geoghegan <pg(at)bowt(dot)ie> |
| Subject: | Re: nbtree: opt-in leaf key prefix compression |
| Date: | 2026-11-18 17:31:49 |
| Message-ID: | 179502032980.1724456.1423341864991075612@burd.me |
| Views: | Whole Thread | Raw Message | Download mbox | Resend email |
| Thread: | |
| Lists: | pgsql-hackers |
Hello,
v13 attached, rebased onto 45047c51aab.
Further testing found that fitting on a page does not guarantee that an
encoded tuple or prefix directory fits its size field. I added
admission checks for those limits, plus checks for final build leaves
whose last item is a posting list. The new tests reproduce both cases
on 32KB builds.
I also found why some incrementally built indexes grew larger with
compression enabled. The coded-page split picker balanced the halves
even on rightmost pages, ignoring fillfactor. Some pages that could
be decoded onto one page also unnecessarily bypassed the ordinary
split picker. The patch now uses that picker when decoding fits and
honors rightmost fillfactor among legal split candidates otherwise.
For the fixed-width path workload, that reduced the 100,000-row index
from 8,118,272 to 5,414,912 bytes; compression off used 5,562,368 bytes.
The attached notes give the other results and reproduction details.
These are size observations, not timing results. Variable-length
headers still limit sharing, and the change does not shrink every
workload or fillfactor setting.
Other changes since v12:
0001: Validate coded metadata before decoding, including raw pageinspect
input, and add corruption tests. Copy amcheck's saved heap TID so the
next decode cannot overwrite a duplicate-key diagnostic. Extend unique
index, NULL, representation-preservation, and standby coverage.
This remains a prototype, but a highly functional one. The fixes do not
settle the format choice or establish that scan CPU costs and
default-off overhead are acceptable.
best.
-greg
| Attachment | Content-Type | Size |
|---|---|---|
| notes.txt | text/plain | 3.0 KB |
| v13-0001-nbtree-add-opt-in-leaf-key-prefix-compression.patch | application/octet-stream | 225.5 KB |
| From | Date | Subject | |
|---|---|---|---|
| Next Message | Bharath Rupireddy | 2026-11-18 17:42:08 | Re: [PATCH] Fall back to serializing when parallel apply runs out of DSM segments |
| Previous Message | Bharath Rupireddy | 2026-11-18 17:29:49 | Re: Parallel vacuum: check XID wraparound failsafe between indexes |