Re: Further reduction of bufmgr lock contention

"Tom Lane" <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote
>
> The thing that's nagging at me at the moment is the realization that a
> partitioned hashtable will eat more shared memory than a single
> hashtable. It wasn't that long ago that we had to do some hacking to
> ensure that the buffer hashtable couldn't run out of memory after
> startup, and I'm afraid of re-introducing that failure mode. The lock
> manager can run out of memory without crashing the system, but the
> bufmgr can't (or at least could not in the recent past...)
>

IHMO overflow is not avoidable no matter we use hash or range. Theoretically
seems we could have a data structure like this: (1) a set of k partition
tables, each is with a LWLock and size NBuffers/k; (2) a set of k overflow
tables (actually we only need k-1) plus a LWLock protecting them, each is
with size NBuffers/k. If any partition table overflows, we can assign a
overflow table for it to contain extra hash elements. At run time, the hash
tables for buffer pool may look like this:

[partition 0]
[partition 1][overflow 2]
[partition 2][overflow 0]
[partition 3]

But I am not sure how difficult to implement it in current hash code -
another handwaiving ...

Regards,
Qingqing