Re: Shrinking TSvectors

On 05/04/2016 15:15, Artur Zakirov wrote:
> On 05.04.2016 14:37, Howard News wrote:
>> Hi,
>>
>> does anyone have any pointers for shrinking tsvectors
>>
>> I have looked at the contents of some of these fields and they contain
>> many details that are not needed. For example...
>>
>> "'+1':935,942 '-0500':72 '-0578':932 '-0667':938 '-266':937 '-873':944
>> '-9972':945 '/partners/application.html':222
>> '/partners/program/program-agreement.pdf':271
>> '/partners/reseller.html':181,1073 '01756':50,1083 '07767':54,1087
>> '1':753,771 '12':366 '14':66 (...)"
>>
>> I am not interested in keeping the numbers or urls in the indexes.
>>
>> Thanks,
>>
>> Howard.
>>
>>
>
> Hello,
>
> You need create a new text search configuration. Here is an example of
> commands:
>
> CREATE TEXT SEARCH CONFIGURATION public.english_cfg (
> PARSER = default
> );
> ALTER TEXT SEARCH CONFIGURATION public.english_cfg
> ALTER MAPPING FOR asciiword, asciihword, hword_asciipart,
> word, hword, hword_part
> WITH pg_catalog.english_stem;
>
> Instead of the "pg_catalog.english_stem" you can use your own dictionary.
>
> Lets compare new configuration with the embedded configuration
> "pg_catalog.english":
>
> postgres=# select to_tsvector('english_cfg', 'home -9972
> /partners/application.html /partners/program/program-agreement.pdf');
> to_tsvector
> -------------
> 'home':1
> (1 row)
>
> postgres=# select to_tsvector('english', 'home -9972
> /partners/application.html /partners/program/program-agreement.pdf');
> to_tsvector
> -----------------------------------------------------------------------------------------------
>
> '-9972':2 '/partners/application.html':3
> '/partners/program/program-agreement.pdf':4 'home':1
> (1 row)
>
>
> You can get some additional information about configurations using \dF+:
>
> postgres=# \dF+ english
> Text search configuration "pg_catalog.english"
> Parser: "pg_catalog.default"
> Token | Dictionaries
> -----------------+--------------
> asciihword | english_stem
> asciiword | english_stem
> email | simple
> file | simple
> float | simple
> host | simple
> hword | english_stem
> hword_asciipart | english_stem
> hword_numpart | simple
> hword_part | english_stem
> int | simple
> numhword | simple
> numword | simple
> sfloat | simple
> uint | simple
> url | simple
> url_path | simple
> version | simple
> word | english_stem
>
> postgres=# \dF+ english_cfg
> Text search configuration "public.english_cfg"
> Parser: "pg_catalog.default"
> Token | Dictionaries
> -----------------+--------------
> asciihword | english_stem
> asciiword | english_stem
> hword | english_stem
> hword_asciipart | english_stem
> hword_part | english_stem
> word | english_stem
>
Thanks Artur,

Thats amazing! Postgres never ceases to amaze me. And the same goes for
the contributors to this list.