Re: Dynamic execution returning large result sets

On 11/06/2016 05:12 PM, Emrul wrote:
> Hi,
>
> I have a function that returns an SQL string as follows:
> CREATE OR REPLACE FUNCTION t1() RETURNS text AS
> $$
> BEGIN
> RETURN 'SELECT * FROM mytable';
> END
> $$ LANGUAGE plpgsql;
>
> and I want to create a second function (t2) that will execute the string
> returned by t1() and return the results. I thought about using RETURN QUERY
> EXECUTE as documented here:
> https://www.postgresql.org/docs/9.6/static/plpgsql-control-structures.html
> but there's a note towards the end that says '/if a PL/pgSQL function
> produces a very large result set, performance might be poor: data will be
> written to disk to avoid memory exhaustion, but the function itself will not
> return until the entire result set has been generated./'
>
> Is there any other way I can achieve execution of the dynamic SQL from t1()
> without having the whole result set retrieved inside the function itself?

https://www.postgresql.org/docs/9.5/static/plpgsql-cursors.html

"Rather than executing a whole query at once, it is possible to set up a
cursor that encapsulates the query, and then read the query result a few
rows at a time. One reason for doing this is to avoid memory overrun
when the result contains a large number of rows. (However, PL/pgSQL
users do not normally need to worry about that, since FOR loops
automatically use a cursor internally to avoid memory problems.) A more
interesting usage is to return a reference to a cursor that a function
has created, allowing the caller to read the rows. This provides an
efficient way to return large row sets from functions."

>
> note: My example above is simplified, my real t1() function takes some
> parameters and generates SQL depending upon those parameters.
>
>
> Thanks!
>
>
>
> --
> View this message in context: http://postgresql.nabble.com/Dynamic-execution-returning-large-result-sets-tp5929177.html
> Sent from the PostgreSQL - general mailing list archive at Nabble.com.
>
>

--
Adrian Klaver
adrian(dot)klaver(at)aklaver(dot)com