Re: storing large amounts of text

Ross J. Reedstrom wrote:

> Just to give everyone here a head's up as to what's coming: The developers
> have long known that the size limit on tuple storage (particularly as it
> affects the 'text' type) needs to go away, and that the existing large
> objects (lo) are less than ideal as a work around. Jan Wieck has done
> the inital plan and development of a generic replacement: the Oversized
> Tuple Storage Technique (TOAST).
>
> http://www.postgresql.org/projects/devel-toast.html

Look at

http://www.postgresql.org/~wieck

for a just uploaded snapshot of the ongoing TOAST
development. It demonstrates with a user defined CLOB type
what the toaster actually can do to store a multiple MB sized
string into a single attribute (of course, one table could
contain several CLOB columns and they get toasted as needed -
biggest first).

> This system allows automatic compression and/or external storage of
> any datatype, directly in the database. It's still in development, but
> Jan has recently released a snapshot, so it's past the initial planning
> stages. For now, we'll all have to make do with lo or external file store,
> but there is hope on the horizon...

AFAIK, we'll finally have a CLOB data type upward compatible
with what's in the above snapshot. So someone in really
urgent need to get going with it please feel free to contact
me directly. The above is not meant to be used in production.
But we should be able to create BLOB in the same way too and
create a useful after 7.0 feature patch in a few weeks.

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#========================================= wieck(at)debis(dot)com (Jan Wieck) #