WIP: Faster Expression Processing v4

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org
Cc: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Doug Doole <ddoole(at)salesforce(dot)com>
Subject: WIP: Faster Expression Processing v4
Date: 2017-02-09 00:33:12
Message-ID: 20170209003312.rr7atqepgbrf4xqe@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers


Here's an updated version of this patchset (now that SRFs are
implemented "properly"). This is a large patchset, with later patches
being more experimental than earlier ones.

0001: Add expression dependencies on composite type / whole row components.
This patch later allows to remove some execution time check (in
ExecEvalScalarVar / ExecEvalFieldSelect), which'd be inconvenient to
do when JITed.

0002: Make get_last_attnums more generic.
What it says.

0003: autoconf test for computed goto support.
The faster expression evaluation implementation uses computed gotos
when available. So we need to check whether the current compiler
supports those.

0004: WIP: Faster expression processing and targetlist projection.
The most important patch of the series. Changes expression evaluation
from being tree-walk based as done in execQual.c, into being a series
of instructions. Then adds an opcode dispatch based interpreter for
that mini language. That allows to later add different
implementations. Projection is now a special case for expression

0005: WIP: Add configure infrastructure to enable LLVM.
0006: WIP: Beginning of a LLVM JIT infrastructure.
Needs work.

0007: WIP: JITed expression evaluation.
This finally performs runtime compilation of expressions.

I think up to 0004 things are getting pretty close to being ready.
There's some minor FIXMEs that need to addressed (primarily around
CaseTestExpr/CoerceToDomainValue when invoked outside of plans), and
some more comment work.

The biggest remaining piece of work in the whole series is 0006. Right
compiled expressions can be leaked, which is obviously not ok. So I need
to write ResourceOwner integration.

0007 covers nearly all types of expressions /queries (function usage
isn't supported right now, and there's two missing calls to
MakeExpandedObjectReadOnly). There's a lot more that could be done for
efficiency (generating smarter LLVM IR basically), but we already see
quite significant performance wins.

I have later patches that
- centralizes tuple deforming logic into slot_deform_logic from
slot_getattr, slot_getsomeatts,slot_getallattrs and such
- add JITing of tuple deforming
- add JITing of aggregation transitions
this works by converting transition functions into expression opcodes,
and then adding JIT support for those new instructions

but the above 7 patches already are large enough imo.

For now 0004 is I think the patch that really could use a look by
somebody but me - it's quite a radical departure from how things
currently look in master, and it's a fairly large patch. I don't really
see a good way to break it down into useful parts :( (the projection
changes are the only thing that I can see usefully being done



Attachment Content-Type Size
0001-Add-expression-dependencies-on-composite-type-whole-.patch.gz application/x-patch-gzip 4.9 KB
0002-Make-get_last_attnums-more-generic.patch.gz application/x-patch-gzip 1.5 KB
0003-Add-autoconf-test-for-computed-goto-support.patch.gz application/x-patch-gzip 1.6 KB
0004-WIP-Faster-expression-processing-and-targetlist-proj.patch.gz application/x-patch-gzip 94.7 KB
0005-WIP-Add-configure-infrastructure-to-enable-LLVM.patch.gz application/x-patch-gzip 2.2 KB
0006-WIP-Beginning-of-a-LLVM-JIT-infrastructure.patch.gz application/x-patch-gzip 3.0 KB
0007-WIP-JITed-expression-evaluation.patch.gz application/x-patch-gzip 11.2 KB

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Yugo Nagata 2017-02-09 01:20:23 Re: Backport of pg_statistics typos fix
Previous Message Tom Lane 2017-02-08 23:54:43 Re: Improve OR conditions on joined columns.