Re: The flinfo->fn_extra question, from me this time.

From: Dent John <denty(at)QQdd(dot)eu>
To: PostgreSQL Hackers <pgsql-hackers(at)postgresql(dot)org>
Cc: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Pavel Stehule <pavel(dot)stehule(at)gmail(dot)com>, Chapman Flack <chap(at)anastigmatix(dot)net>
Subject: Re: The flinfo->fn_extra question, from me this time.
Date: 2019-12-08 20:33:02
Views: Raw Message | Whole Thread | Download mbox | Resend email
Lists: pgsql-hackers

Hi folks,

I’ve updated the patch, addressed the rescan issue, and restructured the tests.

I’ve taken a slightly different approach this time, re-using the (already pipeline-supporting) machinery of the Materialize node, and extended it to allow an SFRM_Materialize SRF to donate the tuplestore it returns. I feel this yields a better code structure, as well getting as more reuse.

It also opens up more informative and transparent EXPLAIN output. For example, the following shows Materialize explicitly, whereas previously a FunctionScan would have silently materialised the result of both generate_series() invocations.

postgres=# explain (analyze, costs off, timing off, summary off)
select * from generate_series(11,15) r, generate_series(11,14) s;
Nested Loop (actual rows=20 loops=1)
-> Function Scan on generate_series s (actual rows=4 loops=1)
-> SRF Scan (actual rows=4 loops=1)
SFRM: ValuePerCall
-> Function Scan on generate_series r (actual rows=5 loops=4)
-> Materialize (actual rows=5 loops=4)
-> SRF Scan (actual rows=5 loops=1)
SFRM: ValuePerCall

I also thought again about when to materialise, and particularly Robert’s suggestion[1] (which is in also this thread, but I didn’t originally understand the implication of). If I’m not wrong, between occasional explicit use of a Materialize node by the planner, and more careful observation of EXEC_FLAG_REWIND and EXEC_FLAG_BACKWARD in FunctionScan’s initialisation, we do actually have what is needed to pipeline without materialisation in at least some cases. There is not a mechanism to preferentially re-execute a SRF rather than materialise it, but because materialisation only seems to be necessary in the face of a join or a scrollable cursor, I’m not considering much of a problem anymore.

The EXPLAIN output needs a bit of work, costing is still a sore point, and it’s not quite as straight-line performant as my first attempt, as well as there undoubtedly being some unanticipated breakages and rough edges.

But the concept seems to work roughly as I intended (i.e., allowing FunctionScan to pipeline). Unless there are any objections, I will push it into the January commit fest for progressing.

(Revised patch attached.)


[1] <>

In response to


Browse pgsql-hackers by date

  From Date Subject
Next Message Tom Lane 2019-12-08 22:08:55 hyrax versus isolationtester.c's hard-wired timeouts
Previous Message Tom Lane 2019-12-08 19:14:03 Re: [Proposal] Level4 Warnings show many shadow vars