Re: JIT compiling - v4.0

From: Andres Freund <andres(at)anarazel(dot)de>
To: pgsql-hackers(at)postgresql(dot)org
Subject: Re: JIT compiling - v4.0
Date: 2017-10-04 06:48:09
Message-ID: 20171004064809.qe45ejsixgmfxd5r@alap3.anarazel.de
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Hi,

Here's an updated version of the patchset. There's some substantial
changes here, but it's still very obviously very far from committable as
a whole. There's some helper commmits that are simple and independent
enough to be committable earlier on.

The git tree of this work, which is *frequently* rebased, is at:
https://git.postgresql.org/gitweb/?p=users/andresfreund/postgres.git;a=shortlog;h=refs/heads/jit

The biggest changes are:

- The JIT "infrastructure" is less bad than before, and starting to
shape up.
- The tuple deforming logic is considerably faster than before due to
various optimizations. The optimizations are:
- build deforming exactly to the required natts for the specific caller
- avoid checking the tuple's natts for attributes that have
"following" NOT NULL columns.
- a bunch of minor codegen improvements.
- The tuple deforming codegen also got simpler by relying on LLVM to
promote a stack variable to a register, instead of working with a
register manually - the need to keep IR in SSA form makes doing so
manually rather painful.
- WIP patch to do execGrouping.c TupleHashTableMatch() via JIT. That
makes the column comparison faster, but more importantly it JITs the
deforming (one side at least always is a MinimalTuple).
- All tests pass with JITed expression, tuple deforming, agg transition
value computation and execGrouping logic. There were a number of bugs,
who would have imagined that.
- some more experimental changes later in the series to address some
bottlenecks.

Functionally this covers all of what I think a sensible goal for v11
is. There's a lot of details to figure out, and the inlining
*implementation* isn't what I think we should do. I'll follow up, not
tonight though, with an email outlining the first few design decisions
we're going to have to finalize, which'll be around the memory/lifetime
management of functions, and other infrastructure pieces (currently
patch 0006).

As the patchset is pretty large already, and not going to get any
smaller, I'll make smaller adjustments solely via the git tree, rather
than full reposts.

Greetings,

Andres Freund

Attachment Content-Type Size
0001-Rely-on-executor-utils-to-build-targetlist-for-DM.v4.patch.gz application/x-patch-gzip 1.0 KB
0002-WIP-Allow-tupleslots-to-have-a-fixed-tupledesc-us.v4.patch.gz application/x-patch-gzip 11.9 KB
0003-Perform-slot-validity-checks-in-a-separate-pass-o.v4.patch.gz application/x-patch-gzip 3.3 KB
0004-Pass-through-PlanState-parent-to-expression-insta.v4.patch.gz application/x-patch-gzip 1.4 KB
0005-Add-configure-infrastructure-to-enable-LLVM.v4.patch.gz application/x-patch-gzip 2.3 KB
0006-Beginning-of-a-LLVM-JIT-infrastructure.v4.patch.gz application/x-patch-gzip 6.8 KB
0007-JIT-compile-expressions.v4.patch.gz application/x-patch-gzip 12.5 KB
0008-Centralize-slot-deforming-logic-a-bit.v4.patch.gz application/x-patch-gzip 2.4 KB
0009-WIP-Make-scan-desc-available-for-all-PlanStates.v4.patch.gz application/x-patch-gzip 789 bytes
0010-JITed-tuple-deforming.v4.patch.gz application/x-patch-gzip 6.8 KB
0011-Simplify-aggregate-code-a-bit.v4.patch.gz application/x-patch-gzip 2.8 KB
0012-More-efficient-AggState-pertrans-iteration.v4.patch.gz application/x-patch-gzip 1.3 KB
0013-Avoid-dereferencing-tts_values-nulls-repeatedly-i.v4.patch.gz application/x-patch-gzip 964 bytes
0014-WIP-Expression-based-agg-transition.v4.patch.gz application/x-patch-gzip 14.7 KB
0015-Hacky-Preliminary-inlining-implementation.v4.patch.gz application/x-patch-gzip 4.9 KB
0016-WIP-Inline-ExecScan-mostly-to-make-profiles-easie.v4.patch.gz application/x-patch-gzip 4.8 KB
0017-WIP-Do-execGrouping.c-via-expression-eval-machine.v4.patch.gz application/x-patch-gzip 4.9 KB
0018-WIP-deduplicate-int-float-overflow-handling-code.v4.patch.gz application/x-patch-gzip 1.9 KB
0019-Make-timestamp_cmp_internal-an-inline-function.v4.patch.gz application/x-patch-gzip 800 bytes
0020-Make-hot-path-of-pg_detoast_datum-an-inline-funct.v4.patch.gz application/x-patch-gzip 880 bytes
0021-WIP-Inline-additional-function.v4.patch.gz application/x-patch-gzip 703 bytes
0022-WIP-Faster-order.v4.patch.gz application/x-patch-gzip 640 bytes

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Laurenz Albe 2017-10-04 06:51:21 Re: list of credits for release notes
Previous Message Fabien COELHO 2017-10-04 06:20:51 Re: PATCH: pgbench - option to build using ppoll() for larger connection counts