Re: [PROPOSAL] Shared Ispell dictionaries

On 21.01.2019 17:56, Tomas Vondra wrote:
> On 1/21/19 12:51 PM, Arthur Zakirov wrote:
>> I'll try to implement the syntax, you suggested earlier:
>>
>> ALTER TEXT SEARCH DICTIONARY x UNLOAD/RELOAD
>>
>> The main point here is that UNLOAD/RELOAD can't release the memory
>> immediately, because some other backend may pin a DSM.
>>
>> The second point we should consider (I think) - how do we know which
>> dictionary should be unloaded. There was such function earlier, which
>> was removed. But what about adding an information in the "\dFd" psql's
>> command output? It could be a column which shows is a dictionary loaded.
>>
> ...The only thing we have is "unload" capability by closing the
> connection...
BTW, even if the connection was closed and there are no other
connections a dictionary still remains "loaded". It is because
dsm_pin_segment() is called during loading the dictionary into DSM.

> ...
> I wonder if we could devise some simple cache eviction policy. We don't
> have any memory limit GUC anymore, but maybe we could use unload
> dictionaries that were unused for sufficient amount of time (a couple of
> minutes or so). Of course, the question is when exactly would it happen
> (it seems far too expensive to invoke on each dict access, and it should
> happen even when the dicts are not accessed at all).

Yes, I thought about such feature too. Agree, it could be expensive
since we need to scan pg_ts_dict table to get list of dictionaries (we
can't scan dshash_table).

I haven't a good solution yet. I just had a thought to return
max_shared_dictionaries_size. Then we can unload dictionaries (and scan
the pg_ts_dict table) that were accessed a lot time ago if we reached
the size limit.
We can't set exact size limit since we can't release the memory
immediately. So max_shared_dictionaries_size can be renamed to
shared_dictionaries_threshold. If it is equal to "0" then PostgreSQL has
unlimited space for dictionaries.

--
Arthur Zakirov
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company