Re: Proposal: efficient iter on named cursors

On Thu, Feb 17, 2011 at 10:44 AM, Federico Di Gregorio
<federico(dot)digregorio(at)dndg(dot)it> wrote:
> On 15/02/11 13:22, Daniele Varrazzo wrote:

>> On this reasoning, I've committed this patch
>> <https://github.com/dvarrazzo/psycopg/commit/1dd71947bff415f1c79a5e6b81b6ba89d717ecb9>
>> in a separate branch: it makes iteration respect arraysize in its
>> default value 1 too, and improves the documentation explaining the
>> complete picture. If ok, the patch will be merged in devel.
>>
>> Comments?
>
> I think the original implementation was right because "foreach ..."
> doesn't mean fetch one record at a time. IMHO,
>
> 1) .fetchone() should _always_ fetch one record
> 2) iter(cursor) should fetch as many records as we feel right

Yes, this is what I think too. It is consistent with what happens with
iter(file) vs. file.readline(). The only hitch is that the DBAPI asks
for a default of 1 for arraysize.

> But we can do a little trick here and make iter(cursor) respect
> .arraysize if arraysize was explicitly set so that if one really wants
> to fetch one record at a time can just set .arraysize to 1.
>
> Good or bad?

Quite tricky as arraysize is currently a simple property. Even if we
could do it with some property trickery, it would be surprising if
"print cur.arraysize" would return 1 and iter(cur) was efficient;
then, after "cur.arraysize = 1", iter(cur) would switch to fetch one
record at time, while "print cur.arraysize" would still report 1. I
feel it violates the principle of least astonishment, as much as being
difficult for the user to predict what the library would do.

-- Daniele