Re: [HACKERS] LZTEXT for rule plan stings

At 10:36 AM 2/26/00 -0500, Bruce Momjian wrote:
>> I see tables that have two or three varchar(4000) columns with other
>> data, i.e. names and stuff that are also varchar but smaller. I know
>> that these don't actually get stuffed with 4000 chars but rather
>> that 4KB is the upper limit of the size of an Oracle varchar and that
>> the author's been lazy. If I had a compressed text or varchar type
>> I'd be quite confident that the application code would run even with
>> an 8KB block size.
>
>Just to clearify, varchar(4000) does not take 4000 chars on disk, while
>char(4000) does use 4000 chars on the disk.

Yes, I know. The point is that without digging into how the code actually
uses these tables, I don't know which, if any, of the columns might
actually get stuffed with two, three, or four thousand characters. If
I'm unlucky, all of them will be. For now my simple solution is to
run with a 16KB blocksize and not worry about it.

This isn't postgresql's fault or whatever, the basic problem is a
lazy web hacker arbitrarily declaring varchar(4000) columns rather
than sitting down and determining what they need, because in Oracle
the amount taken is also only the number of bytes in the string
stuffed into the column.

This is kind of a pointless discussion. We all know that TOAST is
going to be ultra-slick.

lztext was resurrected as an idea by Tom Lane in response to the
explosion in the length of the rule strings generated for views
in PG7.0.

That just triggered a memory on my part that Interbase apparently
compresses their text type, a fact I found interesting enough to
mention.

I'm neither lobbying for or against Postgres implementation of lztext,
text as lztext, or anything else.

I just found the notion interesting...

It would be nice if a simple table/view combination such as I posted
here earlier didn't bomb PG7.0 with a default 8KB blocksize, though!

My own views are working fine since I've switched to a 16KB blocksize
for the reasons hinted at above, but the fact that this example fails
in the default 8KB version is pretty grotty. Tom Lane will probably
have it all fixed via lztext or some other method before most of the
folks on this list read this note :)

Regarding large types, TOAST is clearly the path to follow, and Jan's
plans for TOASTed couples includes compression when appropriate. I
also think we can layer SQL3-compliant BLOBs and CLOBs on top of his
TOAST implementation later on - for compatibility reasons only, of
course.

- Don Baccus, Portland OR <dhogaza(at)pacifier(dot)com>
Nature photos, on-line guides, Pacific Northwest
Rare Bird Alert Service and other goodies at
http://donb.photo.net.