Re: Fastest Index/Algorithm to find similar sentences

Of course, you can use regular expressions and LIKE. Without understanding the structure of your database, I don't know if that can be made efficient. For a collection of sentences, I suspect it would get complicated. It would probably be slow. I guess that what you want to do will be hard to perform in an efficient manner using a standard relational database with commonly used functions such as LIKE and REGEX.

Perhaps one of the bioinformatics projects like PostBIO or PostBIS can be adapted to suit your needs. They deal with quickly finding similar sequences that are very complex, but they are designed specifically for DNA sequences.
Just a thought.

-----Original Message-----
From: pgsql-general-owner(at)postgresql(dot)org [mailto:pgsql-general-owner(at)postgresql(dot)org] On Behalf Of Janek Sendrowski
Sent: Thursday, July 25, 2013 3:55 PM
To: pgsql-general(at)postgresql(dot)org
Subject: [GENERAL] Fastest Index/Algorithm to find similar sentences

Hi,

I'm searching for an algorithm/Index to find similar sentences in a database.

The Fulltextsearch is not really suitable because it doesn't have a tolerance.

The Levenshtein-distance ist to slow.

I also tried pg_trgm module, which works with tri-grams, but it's also very slow with 100.000+ rows.

I hope someone can help, I can't really find sth. which is fast enough.

Best regards,
Janek
 
 

--
Sent via pgsql-general mailing list (pgsql-general(at)postgresql(dot)org) To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-general