Re: Proposal: efficient iter on named cursors

From: Federico Di Gregorio <federico(dot)digregorio(at)dndg(dot)it>
To: Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com>
Cc: psycopg(at)postgresql(dot)org
Subject: Re: Proposal: efficient iter on named cursors
Date: 2011-02-17 10:44:13
Message-ID: 4D5CFBFD.8080103@dndg.it
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: psycopg

On 15/02/11 13:22, Daniele Varrazzo wrote:
> However the ticket #41
> <http://psycopg.lighthouseapp.com/projects/62710/tickets/41>, while
> invalid for me as I don't see "for record in cur" as a synonym for
> "fetch one record at time", has made me wonder if we are too
> aggressive with the default: maybe discarding arraysize=1 is not the
> best option. Albeit somebody can still use the "while True + /
> fetchone() + break" pattern to force record-per-record fetching, I
> think if she is careful enough to use named cursors for its task she
> may also care to set an appropriate value > 1 for arraysize. Named
> cursors are still easier to use, but I don't want to make people think
> they can be a replacement for *all* cursors - they still require more
> resources on the server, so are better used only when required.
>
> On this reasoning, I've committed this patch
> <https://github.com/dvarrazzo/psycopg/commit/1dd71947bff415f1c79a5e6b81b6ba89d717ecb9>
> in a separate branch: it makes iteration respect arraysize in its
> default value 1 too, and improves the documentation explaining the
> complete picture. If ok, the patch will be merged in devel.
>
> Comments?

I think the original implementation was right because "foreach ..."
doesn't mean fetch one record at a time. IMHO,

1) .fetchone() should _always_ fetch one record
2) iter(cursor) should fetch as many records as we feel right

But we can do a little trick here and make iter(cursor) respect
.arraysize if arraysize was explicitly set so that if one really wants
to fetch one record at a time can just set .arraysize to 1.

Good or bad?

federico

--
Federico Di Gregorio federico(dot)digregorio(at)dndg(dot)it
Studio Associato Di Nunzio e Di Gregorio http://dndg.it
I did appreciate the irony that I was whining about encoding issues on
a mailing list that was unable to show those chars, too.
-- Antti S. Lankila to mono-devel-list@

In response to

Responses

Browse psycopg by date

  From Date Subject
Next Message Daniele Varrazzo 2011-02-17 10:57:37 Re: Proposal: efficient iter on named cursors
Previous Message Barend Köbben 2011-02-17 10:05:52 Re: psycopg used in a ASP page fails