Re: Streaming LIBPQ? Sliding Window in LIBPQ?

"Martijn van Oosterhout" <kleptog(at)svana(dot)org> writes:

> On Wed, Nov 28, 2007 at 09:11:07AM -0600, Abraham, Danny wrote:
>> Is there a way to break the PGresult array to chuncks Like Oracle?
>>
>> I mean, without changing the text of given queries,
>>
>> can we somehow limit the memory consumption of a client that is using
>> LIBPQ?
>
> Use non-blocking mode to get the data in chunks. You can't easy control
> the number of rows you get each time though...

This doesn't really help. It's nonblocking but you still can't get libpq to
actually give you a result data structure until the entire results have
accumulated.

The only interface like this libpq supports is to use cursors in your SQL and
then FETCH n for each chunk. There's no libpq support for receiving results
incrementally.

If you're writing a driver implementing the protocol from scratch you could
expose chunks of results to the application but there's no protocol-level
support for it so you can't directly control the rate at which results arrive
or the chunk size or anything like that.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com
Get trained by Bruce Momjian - ask me about EnterpriseDB's PostgreSQL training!