Skip site navigation (1) Skip section navigation (2)

Re: Sphinx indexing problem

From: Mladen Gogala <mladen(dot)gogala(at)vmsinfo(dot)com>
To: Joshua Tolley <eggyknap(at)gmail(dot)com>
Cc: "pgsql-novice(at)postgresql(dot)org" <pgsql-novice(at)postgresql(dot)org>
Subject: Re: Sphinx indexing problem
Date: 2010-05-24 12:02:10
Message-ID: 4BFA6AC2.9020605@vmsinfo.com (view raw or flat)
Thread:
Lists: pgsql-novice
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
  <title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Joshua Tolley wrote:<br>
<blockquote
 cite="mid:AANLkTinR6Fo2M3gAuduV78sqYqPEOw0RX_5-BdbeMdZS(at)mail(dot)gmail(dot)com"
 type="cite">
  <blockquote type="cite">
    <pre wrap="">Is there anything I can do to prevent the API from attempting to put the
entire query result in memory?
    </pre>
  </blockquote>
  <pre wrap=""><!---->
Use a cursor, and fetch chunks of the result set one at a time.

<a class="moz-txt-link-freetext" href="http://www.postgresql.org/docs/current/interactive/sql-declare.html">http://www.postgresql.org/docs/current/interactive/sql-declare.html</a>

--
Joshua Tolley / eggyknap
End Point Corporation

  </pre>
</blockquote>
<br>
I would have done so,&nbsp; had I written the application. Unfortunately,
the application was written by somebody else. Putting the entire result
set in memory is a bad idea and Postgres client should be changed,
probably by adding some configuration options, like maximum memory that
the client is allowed to consume and a "swap file". These options
should be configurable per user, not system-wide. As I have said in my
post, I do have a solution for my immediate problem but this slows
things down:<br>
<br>
<blockquote><tt><font color="#3333ff">[root(at)medo etc]# ../bin/indexer
--all</font></tt><br>
  <tt><font color="#3333ff">Sphinx 0.9.9-release (r2117)</font></tt><br>
  <tt><font color="#3333ff">Copyright (c) 2001-2009, Andrew Aksyonoff</font></tt><br>
  <br>
  <tt><font color="#3333ff">using config file
'/usr/local/etc/sphinx.conf'...</font></tt><br>
  <tt><font color="#3333ff">indexing index 'test1'...</font></tt><br>
  <tt><font color="#3333ff">collected 12757000 docs, 21313.6 MB</font></tt><br>
</blockquote>
<br>
It's still running, from slightly before I sent my post.<br>
<pre class="moz-signature" cols="72">-- 
Mladen Gogala 
Sr. Oracle DBA
1500 Broadway
New York, NY 10036
(212) 329-5251
<a class="moz-txt-link-abbreviated" href="http://www.vmsinfo.com">www.vmsinfo.com</a> 
</pre>
</body>
</html>


Attachment: unknown_filename
Description: text/html (2.1 KB)

In response to

Responses

pgsql-novice by date

Next:From: Marco BombenDate: 2010-05-24 12:09:39
Subject: timestamp of a row
Previous:From: Jayadevan MDate: 2010-05-24 03:40:12
Subject: Re: Best starter book

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group