Re: Restructured Shared Buffer Hash Table

On Wed, Jul 8, 2026 at 12:44 AM Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
>
> On 07/07/2026 21:41, Dhruv Aron wrote:
> > At Databricks, we’ve found that the existing dynahash table structure is
> > leaving performance gains on the table when it comes to shared buffer
> > lookups: the multi-level structure (directory, segment, bucket chain,
> > freelist) appears excessive for the shared buffers and could be
> > simplified to boost performance and lower memory overhead. As such, we
> > are proposing a specialized hash table just for this purpose and would
> > appreciate feedback on this approach.
>
> Yeah, the dynahash has features that we just don't need in the buffer
> lookup table...
>
> The indirection with the directory is actually unnecessary for all the
> shared memory hash tables, since none of them can be resized. I've
> wondered if we should try to eliminate that from dynahash for all shmem
> hash tables. But the buffer lookup table is very performance-critical,
> so if there are any performance gains to be had, it's indeed probably
> worthwhile to have a separate implementation just for it.

For very large buffer pools, the buffer lookup table spans multiple
memory pages. With buffer resizing capability it will be good to be
able to resize the buffer lookup table as well. Right now the patch
does not resize buffer look up table because of its memory layout and
complexity involved in that operation. If we are using a different
data structure for buffer lookup, it will be good to consider ease of
resizing as well.

--
Best Wishes,
Ashutosh Bapat