Re: update on TOAST status'

JanWieck(at)t-online(dot)de (Jan Wieck) writes:
> After debugging something I thought was a bug in the toaster,
> I've found something really causing headaches.
> TOAST AS IS IS NOT CAPABLE OF HOLDING INDEXED VALUES!
> It appears that brtee indices (at least) can keep references
> to old toast values that survive a VACUUM! Seems these
> references live in nodes actually not referring to a heap
> tuple any more, but used during tree traversal in
> comparisions. As if an index tuple delete from a btree not
> necessarily causes the index value to disappear from the
> btree completely. It'll never be returned by an index scan,
> but the value is still there somewhere.

Oooh, nasty. Probably the keys you are looking at are in upper-
level btree pages and indicate the ranges of keys found in lower
pages, rather than being pointers to real tuples.

One answer is to rebuild indexes from scratch during VACUUM,
before we vacuum the TOAST relation. We've been talking about
doing that for a long time. Maybe it's time to bite the bullet
and do it. (Of course that means fixing the relation-versioning
problem, which it seems we don't have a consensus on yet...)

> Seems the designs of btree and toast are colliding. As soon
> as "text" is toastable, this'll hurt - be warned.

Text *is* marked toastable in current CVS...

> AFAICS, we need to detoast values for index inserts allways
> and have another toaster inside the index access methods at
> some day. In the meantime we should decide a safe maximum
> index tuple size and emit an explanative error message on the
> attempt to insert oversized index entries instead of possibly
> corrupting the index.

I don't like that --- seems it would put a definite crimp in the
whole point of TOAST, which is not to have arbitrary limits on field
sizes.

regards, tom lane