Skip site navigation (1) Skip section navigation (2)

Re: Sphinx indexing problem

From: Mladen Gogala <mladen(dot)gogala(at)vmsinfo(dot)com>
To: Joshua Tolley <eggyknap(at)gmail(dot)com>
Cc: "pgsql-novice(at)postgresql(dot)org" <pgsql-novice(at)postgresql(dot)org>
Subject: Re: Sphinx indexing problem
Date: 2010-05-24 12:02:10
Message-ID: (view raw, whole thread or download thread mbox)
Lists: pgsql-novice
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
  <meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
<body bgcolor="#ffffff" text="#000000">
Joshua Tolley wrote:<br>
  <blockquote type="cite">
    <pre wrap="">Is there anything I can do to prevent the API from attempting to put the
entire query result in memory?
  <pre wrap=""><!---->
Use a cursor, and fetch chunks of the result set one at a time.

<a class="moz-txt-link-freetext" href=""></a>

Joshua Tolley / eggyknap
End Point Corporation

I would have done so,&nbsp; had I written the application. Unfortunately,
the application was written by somebody else. Putting the entire result
set in memory is a bad idea and Postgres client should be changed,
probably by adding some configuration options, like maximum memory that
the client is allowed to consume and a "swap file". These options
should be configurable per user, not system-wide. As I have said in my
post, I do have a solution for my immediate problem but this slows
things down:<br>
<blockquote><tt><font color="#3333ff">[root(at)medo etc]# ../bin/indexer
  <tt><font color="#3333ff">Sphinx 0.9.9-release (r2117)</font></tt><br>
  <tt><font color="#3333ff">Copyright (c) 2001-2009, Andrew Aksyonoff</font></tt><br>
  <tt><font color="#3333ff">using config file
  <tt><font color="#3333ff">indexing index 'test1'...</font></tt><br>
  <tt><font color="#3333ff">collected 12757000 docs, 21313.6 MB</font></tt><br>
It's still running, from slightly before I sent my post.<br>
<pre class="moz-signature" cols="72">-- 
Mladen Gogala 
Sr. Oracle DBA
1500 Broadway
New York, NY 10036
(212) 329-5251
<a class="moz-txt-link-abbreviated" href=""></a> 

Attachment: unknown_filename
Description: text/html (2.1 KB)

In response to


pgsql-novice by date

Next:From: Marco BombenDate: 2010-05-24 12:09:39
Subject: timestamp of a row
Previous:From: Jayadevan MDate: 2010-05-24 03:40:12
Subject: Re: Best starter book

Privacy Policy | About PostgreSQL
Copyright © 1996-2017 The PostgreSQL Global Development Group