Re: GIST and TOAST

"Teodor Sigaev" <teodor(at)sigaev(dot)ru> writes:

> I'm afraid that we have some lack of understanding. Flow of operation with
> indexed tuple in gist is:
> - read tuple
> - get n-th attribute with a help of index_getattr
> - call user-defined decompress method which should, at least, detoast value
> - result value is passed to other user-defined method

So when does index_form_tuple get called?

> So, index_form_tuple should toast value, but value is already compressed and
> live in memory. Detoasting of value should be done by decompress method and
> live in memory, and so, only after that value can be passed to other
> user-defined method.

Does every data type define a compress/decompress method? Even if it's not a
data type that normally gets very large?

> As I understand, packing/unpacking varlena header is doing during
> toasting/detoastiong. So, I'm not understand the problem here.

Well we cheated a bit and had heap/index_form_tuple convert the data to packed
format. This saves having to push small tuples through the toaster. So now
tuples can magically become toasted as soon as they go into a tuple even if
they never get pushed through the toaster.

>> There may be places that assume they won't leak detoasted copies of datums. If
>> you could help point those places out they should just need PG_FREE_IF_COPY()
>
> GiST code works in separate memory context to prevent memory leaks. See
> gistinsert/gistbuildCallback/gistfindnext.

So it's perfectly safe to just use DatumGetType and PG_GETARG_TYPE instead of
using DatumGetPointer and PG_GETARG_POINTER and having to manually cast
everywhere, no? It seems like there's a lot of extra pain to maintain the code
in the present style with all the manual casts.

--
Gregory Stark
EnterpriseDB http://www.enterprisedb.com