Re: Introducing coarse grain parallelism by postgres_fdw.

Hello,

> In order to minimize the impact, what can be done is to execute
> fetch_more_data() in asynchronous mode every time, when there only few rows
> left to be consumed. So in current code below
> 1019 /*
> 1020 * Get some more tuples, if we've run out.
> 1021 */
> 1022 if (fsstate->next_tuple >= fsstate->num_tuples)
> 1023 {
> 1024 /* No point in another fetch if we already detected EOF,
> though. */
> 1025 if (!fsstate->eof_reached)
> 1026 fetch_more_data(node, false);
> 1027 /* If we didn't get any tuples, must be end of data. */
> 1028 if (fsstate->next_tuple >= fsstate->num_tuples)
> 1029 return ExecClearTuple(slot);
> 1030 }
>
> replace line 1022 with if (fsstate->next_tuple >= fsstate->num_tuples)
> with if (fsstate->next_tuple >= fsstate->num_tuples -
> SOME_BUFFER_NUMBER_ROWS)
> Other possibility is to call PQsendQuery(conn, sql), after line 2100 and if
> eof_reached is false.
>
> 2096 /* Must be EOF if we didn't get as many tuples as we asked
> for. */
> 2097 fsstate->eof_reached = (numrows < fetch_size);
> 2098
> 2099 PQclear(res);
> 2100 res = NULL;

I see, I'll consider it. If late (lazy) error detection is
allowed, single row mode seems available, too.

regards,

--
Kyotaro Horiguchi
NTT Open Source Software Center