Scanning a large binary field

From: Kynn Jones <kynnjo(at)gmail(dot)com>
To: pgsql-general General <pgsql-general(at)postgresql(dot)org>
Subject: Scanning a large binary field
Date: 2009-03-15 20:42:24
Message-ID: c2350ba40903151342r8dec52cl2834105e0fb44128@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-general

I have a C program that reads a large binary file, and uses the read
information plus some user-supplied arguments to generate an in-memory data
structure that is used during the remainder of the program's execution. I
would like to adapt this code so that it gets the original binary data from
a Pg database rather than a file.
One very nice feature of the original scheme is that the reading of the
original file was done piecemeal, so that the full content of the file
(which is about 0.2GB) was never in memory all at once, which kept the
program's memory footprint nice and small.

Is there any way to replicate this small memory footprint if the program
reads the binary data from a Pg DB instead of from a file?

FWIW, my OS is Linux.

TIA!

Kynn

Responses

Browse pgsql-general by date

  From Date Subject
Next Message John R Pierce 2009-03-15 21:06:37 Re: Scanning a large binary field
Previous Message Oliver Kennedy 2009-03-15 20:25:50 Best-N aggregates