Re: Status report on writeable CTEs

2010/7/17 Marko Tiikkaja <marko(dot)tiikkaja(at)cs(dot)helsinki(dot)fi>:
> On 7/16/10 6:15 PM +0300, Hitoshi Harada wrote:
>>
>> 1. Use MaterialNode instead of adding DtScanNode. Since MaterialNode
>> is exsiting one that work with single tuplestore, it might be sane to
>> modify this so that it accepts tuplestore from Query instead of its
>> child node.
>
> I thought about this, but I don't necessarily like the idea of overloading
> executor nodes.

Neither do I have good shape for this solution. Maybe it's not good
idea. But my concern is adding DtScanNode, which looks similar to
MaterialNode. Of course each purpose is different, but quite big part
will overlap each other, I think.

>> 2. Use temp table instead of tuplestore list. Since we agreed we need
>> to execute each plan one by one starting and shutting down executor,
>> it now looks very simple strategy.
>
> I didn't look at this because I thought using a "tuplestore receiver" in the
> portal logic was simple enough.  Any thoughts on how this would work?

It's just deconstructing queries like:

WITH t AS (INSERT INTO x ... RETURING *)
SELECT * FROM t;

to

CREATE TEMP TABLE t AS INSERT INTO x ... RETURING *;
SELECT * FROM t;

While the second statement is not implemented yet, it will be so simpler.

Another concern is tuplestore's memory exhausting. Tuplestore holds
tuples in memory as far as the estimated memory usage is within
work_mem (for *each* not total of all tuplestores!), but if you create
dozens of tuplestore (and it's quite possible in wCTE use cases) we
will possibly fail into memory overflow problems.

>> I'm not familiar with the long discussion on this feature so not sure
>> they are possible, but ISTM  they are enough to be discussed (or
>> discussed already?).
>
> We haven't discussed this part of the design yet..  Now is a good time to do
> it.

Yeah, we should. Anyone has another idea? Or adding DtScanNode for
this features is fair enough?

Regards,

--
Hitoshi Harada