Re: index structure for 114-dimension vector

From: Mark Kirkwood <markir(at)paradise(dot)net(dot)nz>
To: Andrew Lazarus <andrew(at)pillette(dot)com>
Cc: pgsql-performance(at)postgresql(dot)org
Subject: Re: index structure for 114-dimension vector
Date: 2007-04-21 00:42:35
Message-ID: 46295DFB.7020607@paradise.net.nz
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-performance

Andrew Lazarus wrote:
> Because I know the 25 closest are going to be fairly close in each
> coordinate, I did try a multicolumn index on the last 6 columns and
> used a +/- 0.1 or 0.2 tolerance on each. (The 25 best are very probably inside
> that hypercube on the distribution of data in question.)
>
> This hypercube tended to have 10-20K records, and took at least 4
> seconds to retrieve. I was a little surprised by how long that took.
> So I'm wondering if my data representation is off the wall.
>
> I should mention I also tried a cube index using gist on all 114
> elements, but CREATE INDEX hadn't finished in 36 hours, when I killed
> it, and I wasn't in retrospect sure an index that took something like
> 6GB by itself would be helpful on a 2GB of RAM box.
>
> MK> I don't think that will work for the vector norm i.e:
>
> MK> |x - y| = sqrt(sum over j ((x[j] - y[j])^2))
>
>

Sorry, in that case it probably *is* worth trying out 6 single column
indexes and seeing if they get bitmap and'ed together...

Mark

In response to

Browse pgsql-performance by date

  From Date Subject
Next Message Tom Lane 2007-04-21 00:44:31 Re: index structure for 114-dimension vector
Previous Message Andrew Lazarus 2007-04-21 00:28:20 Re: index structure for 114-dimension vector