From: | John Naylor <johncnaylorls(at)gmail(dot)com> |
---|---|
To: | PostgreSQL Hackers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | use radix tree for bitmap heap scan |
Date: | 2025-07-03 08:43:52 |
Message-ID: | CANWCAZaKLii6bDV_ZBP05jQLdFVD0d0Yjrfp6c9a6THFfXiKAg@mail.gmail.com |
Views: | Whole Thread | Raw Message | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
This is not in reviewable state, but I wanted to get a sketch of a PoC
out there and highlight some issues that will need to be tackled. The
index scan successfully populates the radix tree and the contents are
verified to match the hash table for the queries present in the
regression tests. (Now that I think of it, this patch set might not be
buildable as is, since my local tree has two copies of the radix tree
template so TID store doesn't break while working on this)
First issue: Different memory management needs from vacuum. I split
RT_SET into two functions, the lower of which is similar to simple
hash's "insert" method.
Second issue: TBM intersection
1) Radix tree iteration assumes that no one will delete keys out from
under it. That includes the caller itself! This is not sufficient for
intersecting bitmaps. I kluged around it by saving
blocks-to-be-deleted in a stack array, but a proper solution could be
to (vigorous handwaving)
1) save per-leaf-node keys in the iterator
2) delete the keys when iteration leaves that node, possibly with a
bulk-delete mechanism
3) restart iteration from the top, at the next part of the key space
The above is enough to get rid of the qsort call, but next we'll want
to use variable-length bitmaps like vacuum does. This is both to
reduce memory use and to allow the length of the line pointer array to
be sized independently from MaxHeapTuplesPerPage. That will require
getting rid of the iterator array and doing shared iteration over the
tree itself. Ideally, this will be done in a way compatible with
vacuum, but resolving the differences into new abstractions will be
the final step.
--
John Naylor
Amazon Web Services
Attachment | Content-Type | Size |
---|---|---|
v1-0001-Get-rid-of-TBMStatus.patch | text/x-patch | 8.7 KB |
v1-0005-Populate-radix-tree-at-the-same-time-as-hash-tabl.patch | text/x-patch | 21.6 KB |
v1-0002-TEMP-disable-tbm_lossify.patch | text/x-patch | 844 bytes |
v1-0003-Turn-RT_SET-into-wrapper-that-calls-to-RT_GET_SLO.patch | text/x-patch | 4.2 KB |
v1-0004-Get-rid-of-RT_CHILD_PTR.patch | text/x-patch | 35.4 KB |
From | Date | Subject | |
---|---|---|---|
Next Message | Andrei Lepikhov | 2025-07-03 09:08:54 | Re: Reduce "Var IS [NOT] NULL" quals during constant folding |
Previous Message | Álvaro Herrera | 2025-07-03 08:19:52 | Re: Inconsistent LSN format in pg_waldump output |