Re: WIP: shared ispell dictionary

2010/3/18 Heikki Linnakangas <heikki(dot)linnakangas(at)enterprisedb(dot)com>:
> Pavel Stehule wrote:
>> attached patch add possibility to share ispell dictionary between
>> processes. The reason for this is the slowness of first tsearch query
>> and size of allocated memory per process. When I tested loading of
>> ispell dictionary (for Czech language) I got about 500 ms and 48MB.
>> With simple allocator it uses only 25 MB. If we remove some check and
>> tolower string transformation from loading stage it needs only 200 ms.
>> But with broken dict or affix file it can put wrong results. This
>> patch significantly reduce load on servers that use ispell
>> dictionaries.
>>
>> I know so Tom worries about using of share memory. I think so it
>> unnecessarily. After loading data from dictionary are only read, never
>> modified. Second idea - this dictionary template can be distributed as
>> separate project (it needs a few changes in core - and simple
>> allocator).
>
> Fixed-size shared memory blocks are always problematic. Would it be
> possible to do the preloading with shared_preload_libraries somehow?

Maybe. But there are some disadvantages: a) you have to copy
dictionary info to config, b) on some systems can be a problem lot of
memory per process (probably not on linux). Still you have to do some
bridge between tsearch cache and preloaded data.

Pavel

>
> --
>  Heikki Linnakangas
>  EnterpriseDB   http://www.enterprisedb.com
>