Re: Faster methods for getting SPI results (460% improvement)

From: Jim Nasby <Jim(dot)Nasby(at)BlueTreble(dot)com>
To: Craig Ringer <craig(at)2ndquadrant(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>, Joe Conway <mail(at)joeconway(dot)com>
Subject: Re: Faster methods for getting SPI results (460% improvement)
Date: 2017-03-01 05:42:44
Message-ID: 06b8a6c0-e496-3565-1695-e55dfeba5e65@BlueTreble.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 1/24/17 10:43 PM, Jim Nasby wrote:
>
>> I strongly suggest making this design effort a separate thread, and
>> focusing on the SPI improvements that give "free" no-user-action
>> performance boosts here.
>
> Fair enough. I posted the SPI portion of that yesterday. That should be
> useful for pl/R and possibly pl/perl. pl/tcl could make use of it, but
> it would end up executing arbitrary tcl code in the middle of portal
> execution, which doesn't strike me as a great idea. Unfortunately, I
> don't think plpgsql could make much use of this for similar reasons.
>
> I'll post a plpython patch that doesn't add the output format control.

I've attached the results of that. Unfortunately the speed improvement
is only 27% at this point (with 9999999 tuples). Presumably that's
because it's constructing a brand new dictionary from scratch for each
tuple.
--
Jim Nasby, Data Architect, Blue Treble Consulting, Austin TX
Experts in Analytics, Data Architecture and PostgreSQL
Data in Trouble? Get it in Treble! http://BlueTreble.com
855-TREBLE2 (855-873-2532)

Attachment Content-Type Size
0001-Add-SPI_execute_callback-and-callback-based-DestRece.patch text/plain 8.3 KB
0002-Minimal-adoption-of-SPI-callbacks-in-plpython.patch text/plain 13.0 KB

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Amit Kapila 2017-03-01 05:43:25 Re: Proposal : Parallel Merge Join
Previous Message Kuntal Ghosh 2017-03-01 05:35:33 Re: Performance degradation in TPC-H Q18