Re: Redundant tuple copy in tqueueReceiveSlot()

Hi,

On 2020-09-17 14:20:50 +1200, Thomas Munro wrote:
> I wonder if there is a way we could make "Minimal Tuples but with the
> length travelling separately (and perhaps chopped off?)" into a
> first-class concept... It's also a shame to be schlepping a bunch of
> padding bytes around.

There really is no justification for having MinimalTuples, as we have
them today at least, anymore. We used to rely on being able to construct
pointers to MinimalTuples that are mostly compatible with HeapTuple. But
I think there's none of those left since PG 12.

I think it'd make a bit more sense to do some steps towards having a
more suitable "minimal" tuple representation, rather than doing this
local, pretty ugly, hacks. A good way would be to just starting to
remove the padding, unnecessary fields etc from MinimalTuple.

I also think that it'd be good to look at a few of the other places that
are heavily bottlenecked by MinimalTuple overhead before designing new
API around this. IIRC it's e.g. very easy to see hash joins spending a
lot of time doing MinimalTuple copies & conversions.

>
> tuple = (MinimalTuple) data;
> - Assert(tuple->t_len == nbytes);
> + tuple->t_len = nbytes;
>
> Hmm, so you have to scribble on shared memory on the receiving side.

Ick, I would really like to avoid this.

> > Thomas, I guess you had a different approach in mind when you said
> > about "returning either success or
> > hey-that-buffer's-too-small-I-need-N-bytes". But what looks clear to
>
> Yeah I tried some things like that, but I wasn't satisfied with any of
> them; basically the extra work involved in negotiating the size was a
> bit too high. On the other hand, because my interface was "please
> write a MinimalTuple here!", it had the option to *form* a
> MinimalTuple directly in place, whereas your approach can only avoid
> creating and destroying a temporary tuple when the source is a heap
> tuple.

There's a lot of cases where the source is a virtual slot (since we'll
often project stuff below Gather). So it'd be quite advantageous to
avoid building an unnecessary HeapTuple in those cases.

I wonder if it would be sensible to build minimal tuples using
tts_values/isnull in some cases. This might even be advantageous in
case of heap / minimal tuples, because IIRC right now the code will
materialize the slot unnecessarily. Not sure how common that is.

Greetings,

Andres Freund