From: | Sean Davis <sdavis2(at)mail(dot)nih(dot)gov> |
---|---|
To: | "mladen(dot)gogala(at)vmsinfo(dot)com" <mladen(dot)gogala(at)vmsinfo(dot)com> |
Cc: | Joshua Tolley <eggyknap(at)gmail(dot)com>, "pgsql-novice(at)postgresql(dot)org" <pgsql-novice(at)postgresql(dot)org> |
Subject: | Re: Sphinx indexing problem |
Date: | 2010-05-24 13:33:35 |
Message-ID: | AANLkTik1ka95EP1kxKC83-46fVwq5UHOj5SiJtwHLi4Z@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-novice |
On Mon, May 24, 2010 at 8:02 AM, Mladen Gogala <mladen(dot)gogala(at)vmsinfo(dot)com>wrote:
> Joshua Tolley wrote:
>
> Is there anything I can do to prevent the API from attempting to put the
> entire query result in memory?
>
>
> Use a cursor, and fetch chunks of the result set one at a time.
> http://www.postgresql.org/docs/current/interactive/sql-declare.html
>
> --
> Joshua Tolley / eggyknap
> End Point Corporation
>
>
>
>
> I would have done so, had I written the application. Unfortunately, the
> application was written by somebody else. Putting the entire result set in
> memory is a bad idea and Postgres client should be changed, probably by
> adding some configuration options, like maximum memory that the client is
> allowed to consume and a "swap file". These options should be configurable
> per user, not system-wide. As I have said in my post, I do have a solution
> for my immediate problem but this slows things down:
>
> [root(at)medo etc]# ../bin/indexer --all
> Sphinx 0.9.9-release (r2117)
> Copyright (c) 2001-2009, Andrew Aksyonoff
>
> using config file '/usr/local/etc/sphinx.conf'...
> indexing index 'test1'...
> collected 12757000 docs, 21313.6 MB
>
>
> It's still running, from slightly before I sent my post.
>
Sounds like a case for talking to the sphinx community?
Sean
From | Date | Subject | |
---|---|---|---|
Next Message | Frank Bax | 2010-05-24 21:03:08 | Re: GPS positions |
Previous Message | Joshua Tolley | 2010-05-24 12:36:46 | Re: Sphinx indexing problem |