Re: patch: preload dictionary new version

2010/7/8 Robert Haas <robertmhaas(at)gmail(dot)com>:
> On Wed, Jul 7, 2010 at 10:50 PM, Takahiro Itagaki
> <itagaki(dot)takahiro(at)oss(dot)ntt(dot)co(dot)jp> wrote:
>> This patch allocates memory with non-file-based mmap() to preload text search
>> dictionary files at the server start. Note that dist files are not mmap'ed
>> directly in the patch; mmap() is used for reallocatable shared memory.
>
> I thought someone (Tom?) had proposed idea previously of writing a
> dictionary precompiler that would produce a file which could then be
> mmap()'d into the backend.  Has any thought been given to that
> approach?

The precompiler can save only some time related to parsing. But it
isn't main issue. Without simple allocation the data from dictionary
takes about 55 MB, with simple allocation about 10 MB. If you have a
100 max_session, then these data can be 100 x repeated in memory -
about 1G (for Czech dictionary). I think so memory can be used
better.

Minimally you have to read these 10MB from disc - maybe from file
cache - but it takes some time too - but it will be significantly
better than now.

Regards
Pavel Stehule

>
> --
> Robert Haas
> EnterpriseDB: http://www.enterprisedb.com
> The Enterprise Postgres Company
>