Re: Searching GIN-index (FTS) and sort by timestamp-column

På mandag 21. mars 2016 kl. 16:13:07, skrev Oleg Bartunov <obartunov(at)gmail(dot)com
<mailto:obartunov(at)gmail(dot)com>>:
    On Mon, Mar 21, 2016 at 5:41 PM, Andreas Joseph Krogh <andreas(at)visena(dot)com
<mailto:andreas(at)visena(dot)com>> wrote: På lørdag 19. mars 2016 kl. 03:44:55, skrev
Jeff Janes <jeff(dot)janes(at)gmail(dot)com <mailto:jeff(dot)janes(at)gmail(dot)com>>:
On Wed, Mar 16, 2016 at 6:53 AM, Andreas Joseph Krogh <andreas(at)visena(dot)com
<mailto:andreas(at)visena(dot)com>> wrote: På onsdag 16. mars 2016 kl. 14:37:27, skrev
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us <mailto:tgl(at)sss(dot)pgh(dot)pa(dot)us>>:
Andreas Joseph Krogh <andreas(at)visena(dot)com <mailto:andreas(at)visena(dot)com>> writes:
> 1. Why isnt' folder_id part of the index-cond?

Because a GIN index is useless for sorting.

> 2. Is there a way to make it use the (same) index to sort by
> received_timestamp?

No.

> 3. Using a GIN-index, is there a way to use the index at all for sorting?

No.

> 4. It doesn't seem like ts_rank uses the index for sorting either.

Same reason.

regards, tom lane
 
So it's basically impossible to use FTS/GIN with sorting on large datasets?
Are there any plans to improve this situation?
 
I don't see why it would not be possible to create a new execution node type
that does an index scan to obtain order (or just to satisfy an equality or
range expression), and takes a bitmap (as produced by the FTS/GIN) to apply as
a filter.  But, I don't know of anyone planning on doing that.

 
Isn't this what Postgres Pro are planning?
http://postgrespro.com/roadmap/mssearch
<http://postgrespro.com/roadmap/mssearch>
 
"Unlike external special-purpose search engines, a full-text search engine
built in a DBMS is capable of combining full-text and attributive search
criteria in SQL query syntax. It is planned to improve the existing PostgreSQL
full-text search engine byextending the functionality of Generalized Inverted
Index (GIN) to make it capable of storing extra information required for
ranging query results. This search acceleration will allow to go back from
external full-text search engines, thus facilitating system administration and
use, reducing technology risks, and improving information security."
 
This is different feature ! Actually, we already have prototype of what Jeff
suggested, we called it bitmap filtering, but failed to find use case where it
provides benefits. Teodor will comment this idea more detail.

 
The feature I'm missing is the ability to do FTS (or use GIN in general) and
then sort on some other column (also indexed by the same GIN-index, using the
btree-gin extention), often of type BIGINT or TIMESTAMP.
Are you planning to work on such a feature for GIN?
 
Thanks.
 
-- Andreas Joseph Krogh
CTO / Partner - Visena AS
Mobile: +47 909 56 963
andreas(at)visena(dot)com <mailto:andreas(at)visena(dot)com>
www.visena.com <https://www.visena.com>
<https://www.visena.com>