Re: update on TOAST status'

Tom Lane wrote:
> JanWieck(at)t-online(dot)de (Jan Wieck) writes:
> > ... So do we want to have indices storing plain
> > values allways and limit them in the index-tuple size or not?
>
> I think not: it will be seen as a robustness failure, even (or
> especially) if it doesn't happen often. I can see the bug reports now:
> "Hey! I tried to insert a long value in my field, and it didn't work!
> I thought you'd fixed this bug?"
>
> You make good arguments that we shouldn't be too concerned about the
> speed of access to toasted index values, and I'm willing to accept
> that point of view (at least till we have hard evidence about it).
> But when I say "it should be bulletproof" I mean it should *work*,
> without imposing arbitrary limits on the user. Arbitrary limits are
> exactly what we are trying to eliminate.

After debugging something I thought was a bug in the toaster,
I've found something really causing headaches.

TOAST AS IS IS NOT CAPABLE OF HOLDING INDEXED VALUES!

It appears that brtee indices (at least) can keep references
to old toast values that survive a VACUUM! Seems these
references live in nodes actually not referring to a heap
tuple any more, but used during tree traversal in
comparisions. As if an index tuple delete from a btree not
necessarily causes the index value to disappear from the
btree completely. It'll never be returned by an index scan,
but the value is still there somewhere.

Everything is OK with this up to a VACUUM run. The toaster
uses SnapShotAny to fetch toast values. So an external value
can be fetched by the toaster even if it is already deleted
and committed. If he has a reference somewhere, he has
allways a share or higher lock on the main relation
preventing VACUUM to mangle up the toast relation (I moved
toast relation vacuuming into the lock time of the main table
recently).

But in the above case it is already vacuumed and not present
any more. Now the btree traversal needs to compare against a
value, long gone to the bit heaven, and that cannot work with
the toast architecture.

Seems the designs of btree and toast are colliding. As soon
as "text" is toastable, this'll hurt - be warned.

AFAICS, we need to detoast values for index inserts allways
and have another toaster inside the index access methods at
some day. In the meantime we should decide a safe maximum
index tuple size and emit an explanative error message on the
attempt to insert oversized index entries instead of possibly
corrupting the index.

Comment!

Jan

--

#======================================================================#
# It's easier to get forgiveness for being wrong than for being right. #
# Let's break this rule - forgive me. #
#================================================== JanWieck(at)Yahoo(dot)com #