Re: Compressed TOAST Slicing

Paul Ramsey <pramsey(at)cleverelephant(dot)ca> writes:
> On Thu, Nov 1, 2018 at 2:29 PM Stephen Frost <sfrost(at)snowman(dot)net> wrote:
>> and secondly, why we wouldn't consider
>> handling a non-zero offset. A non-zero offset would, of course, still
>> require decompressing from the start and then just throwing away what we
>> skip over, but we're going to be doing that anyway, aren't we? Why not
>> stop when we get to the end, at least, and save ourselves the trouble of
>> decompressing the rest and then throwing it away.

> I was worried about changing the pg_lz code too much because it scared me,
> but debugging some stuff made me read it more closely so I fear it less
> now, and doing interior slices seems not unreasonable, so I will give it a
> try.

I think Stephen was just envisioning decompressing from offset 0 up to
the end of what's needed, and then discarding any data before the start
of what's needed; at least, that's what'd occurred to me. It sounds like
you were thinking about hacking pg_lz to not write the leading data
anywhere. While that'd likely be a win for cases where there was leading
data to discard, I'm worried about adding any cycles to the inner loops
of the decompressor. We don't want to pessimize every other use of pg_lz
to buy a little bit for these cases.

regards, tom lane