Re: Significantly larger toast tables on 8.4?

On Saturday 03 January 2009 03:36:16 Tom Lane wrote:
> "Stephen R. van den Berg" <srb(at)cuci(dot)nl> writes:
> > - I currently have difficulty imagining applications that actually do
> > lots of substring extractions from large compressible fields.
>
> The code that's in there to make this happen was written by people who
> needed the feature. They're going to be upset with you if you propose
> disabling it.

I think what he is saying is that it is the less likely use case and should
therefore tend to be not the default.

Also note that the code in there was written about 8 years ago, when dealing
with "large" data was an entirely different game. People where happy to
access more than 8 kB then.

I would in fact imagine that substring operations are more likely to happen
with data smaller than 1 MB, and less likely with data larger than 1 MB,
instead of the other way around, which is currently implemented. The main
sensible way to access text fields larger than 1 MB is with text search, as
was pointed out. And large bytea fields are probably media files that are
probably already compressed and have no sensible use for substring
operations.