Re: TupleTableSlot abstraction

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org, Robert Haas <robertmhaas(at)gmail(dot)com>, Haribabu Kommi <kommi(dot)haribabu(at)gmail(dot)com>, Amit Khandekar <amitdkhan(dot)pg(at)gmail(dot)com>
Cc: Alvaro Herrera <alvherre(at)2ndquadrant(dot)com>, Amit Kapila <amit(dot)kapila16(at)gmail(dot)com>
Subject: Re: TupleTableSlot abstraction
Date: 2018-11-13 23:30:21
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


Attached is a heavily revised version of the patchset developed in this
thread. It's based on the last version posted by Amit Khandekar at [1].

The largest issue this resolves is that, for JITing, we need to know
what kind of slot we're going to deal with. E.g. in contrast to right
now, we do not want to generate bespoke deform function for virtual
slots, and we need to be able to discern between slots we know to deform
and slots where we don't. Up to now generated a deform function
whenever we can figure out what the tupledesc for an EEOP_*_FETCHSOME
operation is, using the fairly ugly hack for looking at
PlanState->scanslot the scan desc, and the right/left tree's for
INNER/OUTER. That kind of works as far as descriptors go, but doesn't
for the slot types - it's far from uncommon that the slot type inside a
node differs from the result type from the left/right tree.

I've tried various approaches for this, and what I settled on is that
every PlanState signals in new fields what its scan result type and slot
type is, if known; normally that's automatic due to using
ExecInitScanTupleSlot(), ExecInitResultSlot(). By default, the right /
left (aka inner/outer) are inferred using ExecGetResultSlotOps on the
subsidiary node, but can be overriden too. This gets rid of similar
logic previously present in llvmjit_expr.c, by moving the relevant
knowledge into ExprEvalOp.d.fetch.

While this primarily benefits JITing, it also allows for future
optimizations like eliding slot_getsomattr() / EEOP_*_FETCHSOME calls
for Vars on slots known to be virtual (which is a large percentage, due
to ExecProject() etc).

To avoid bugs around this, I've added assertions that check that hte
slot type is as expected. This, unsurprisingly, caught a number of bugs.

While this approach isn't perfect (in particular, it adds a few new
fields to PlanState), I've tried hard ot find other solutions, and
didn't come up with anything that's meaningfully better. If anybody has
better ideas, please call now.

The second bigger change is that I've removed ExecFetchGenericSlotTuple
- I really disliked having both that and ExecFetchSlotTuple. Now
ExecFetchSlotTuple has a new *shouldFree parameter that indicates
whether the returned value is allocated in the current context, or
not. I've also renamed it to ExecFetchSlotHeapTuple. It's currently
still used for places where we modify the returned tuple in-place (to
set oid etc), but subsequent patches in the pluggable storage patchset,
get rid of that too.

I've done a bit of benchmarking (TPCH and pgench), and after some
performance fixes the patchset either has no performance effect, or a
very small performance benefit (when using JITing, it reduces the

Further changes:
- changed the split of patches, so they now all pass the
tests, and individually make (some) sense.
- virtual tuple table slots can now be materialized (by serializing
!byval columns into a single allocation, that's then pointed to by
- The responsibility for filling missing columns is now again in
slot_getsomeattrs, rather than the per-slot callbacks (which can still
do so if desired, but normally I don't see much point).
- lots of bugfixes
- comment cleanups
- performance improvements
- Reduction in the number of callbacks. getattr, attisnull are gone.

What I'm now planning to do is to go through the big comment in
tuptable.h and update that to the new world. While I'm not convinced
that that that's the best place for it, it needs to be accurate.

- More comment polishing
- I'll probably split the commits up a bit further (particulary JIT
ignoring virtual tuple slots, inlining the hot path of
- serious commit message polishing

After this, I hope Amit Khandekar will rebase a patch he's sent me
internally that converts triggers to use slots. I'll work on rebasing
the pluggable storage patch ontop of this.



Andres Freund

Attachment Content-Type Size
v14-0001-Rejigger-materializing-and-fetching-a-HeapTuple-.patch text/x-diff 26.7 KB
v14-0002-Change-tuple-table-slot-creation-routines-to-sui.patch text/x-diff 81.2 KB
v14-0003-Use-separate-tuple-table-slot-to-return-tuples-f.patch text/x-diff 3.7 KB
v14-0004-Compute-information-about-EEOP_-_FETCHSOME-at-ex.patch text/x-diff 8.3 KB
v14-0005-Verify-that-expected-slot-types-match-returned-s.patch text/x-diff 5.3 KB
v14-0006-Restructure-TupleTableSlot-to-allow-tuples-other.patch text/x-diff 96.1 KB
v14-0007-Rationalize-expression-context-reset-in-ExecModi.patch text/x-diff 2.4 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Thomas Munro 2018-11-13 23:43:59 Re: Refactoring the checkpointer's fsync request queue
Previous Message Tom Lane 2018-11-13 23:20:30 Re: date_trunc() in a specific time zone