Re: Number of buckets/partitions of dshash

Hello.

At Fri, 12 Oct 2018 06:19:12 +0000, "Ideriha, Takeshi" <ideriha(dot)takeshi(at)jp(dot)fujitsu(dot)com> wrote in <4E72940DA2BF16479384A86D54D0988A6F1C1779(at)G01JPEXMBKW04>
> Hi, let me clarify my understanding about the $title.
>
> It seems that the number of hash partitions is fixed at 128 in dshash and
> right now we cannot change it unless dshash.c code itself is changed, right?
>
> According to the comment of dshash.c, DSHASH_NUM_PARTITIONS could be runtime parameter in future.
> Are there someone working on it?
> (I want to do it, but TBH I cannot promise it because of some other work.)
>
> In my current development for allocating catalog cache on the shared memory[1]
> I'm trying to put hash table of each CatCache to the shared memory using dshash.
> The number of buckets for CatCache is pre-defined by cacheinfo and most of them is under 128 like 8 or 16.
> This would cause some waste of memory on DSA because some partitions (buckets) is allocated but not used.
>
> So I'm thinking that current dshash design is still ok but flexible size of partition is appropriate
> for some use cases like mine.
>
> Do you have any thoughts?

We could do that easily, but shouldn't we find a way to reduce or
eliminate the impact of locking first? dshash needs to hold
partition lock while the caller is examining a returned entry.

> [1] https://www.postgresql.org/message-id/4E72940DA2BF16479384A86D54D0988A567B9245%40G01JPEXMBKW04

regards.

--
Kyotaro Horiguchi
NTT Open Source Software Center