Re: Faster methods for getting SPI results (460% improvement)

On 2017-04-06 21:06:59 -0700, Jim Nasby wrote:
> On 4/6/17 9:04 PM, Andres Freund wrote:
> > On 2017-04-06 09:14:43 -0700, Jim Nasby wrote:
> > > On 4/6/17 9:04 AM, Peter Eisentraut wrote:
> > > > On 4/6/17 03:50, Craig Ringer wrote:
> > > > > But otherwise, pending docs changes, I think it's ready for committer.
> > > >
> > > > My opinion is still that this is ultimately the wrong approach. The
> > > > right fix for performance issues in PL/Python is to change PL/Python not
> > > > to materialize the list of tuples. Now with this change we would be
> > > > moving from two result materializations to one, but I think we are
> > > > keeping the wrong one.
> > >
> > > That's an option for future improvement, but I see no way to accomplish that
> > > without completely breaking plpy.
> >
> > Why? We could very well return a somewhat "smarter" object. Returning
> > rows row-by-row if accessed via iterator, materializes when accessed via
> > row offset.
>
> I completely agree with that. What I don't understand is the objection to
> speeding up the old access method. Or are you thinking we'd just abandon the
> old method?

What I'm saying is that we can do that transparently, with the current
API. And there's no need to materialize anything in plpython, we can
transparently use the SPI materialized version.

Greetings,

Andres Freund