Re: psycopg and gevent with pgbouncer

On Thu, Oct 6, 2011 at 3:53 PM, Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com
> wrote:

> On Thu, Oct 6, 2011 at 12:41 PM, Raoul Duke <knevcher(at)gmail(dot)com> wrote:
>
> > I use psycopgreen for gevent and psycopg2 and it works fine.
>
> Good. Note that you haven't included the mailing list in your reply.
> If your problem is solved, let it know to other who may stumble in
> your request via google.
>

I don't understand. How can I do that?

>
> > Is there any way to detect if my application reconnect to db (pgbouncer)
> on
> > each client request? I use psygopg2.pool.SimpleConnectionPool.
>
> Why do you need the psycopg2 pool if you are using pgbouncer? It
> shouldn't save you a lot.
>

I use pooler to limit requests to DB. If decide not use pgbouncer I think
pooler is required.

> The simple connection pool is not appropriate for concurrent programs:
> you should use probably the ThreadedConnectionPool, after checking
> that the lock it uses is correctly made green by gevent
>

I use this tool https://github.com/tellapart/gevent_request_profiler to
check if my app has locks.

On Thu, Oct 6, 2011 at 2:31 PM, Daniele Varrazzo <daniele(dot)varrazzo(at)gmail(dot)com
> wrote:

> On Thu, Oct 6, 2011 at 9:54 AM, Marko Kreen <markokr(at)gmail(dot)com> wrote:
> > On Thu, Oct 6, 2011 at 10:45 AM, Raoul Duke <knevcher(at)gmail(dot)com> wrote:
>
> >> 2. I use psycopg simple connection pooler :
> >> pool = SimpleConnectionPool(1, 3, connection_string)
> >>
> >> if I add async option:
> >> pool = SimpleConnectionPool(1, 3, connection_string, async=1)
> >>
> >> I've got an error, than I can't understand:
> >> File "/usr/lib/pymodules/python2.6/sqlalchemy/engine/base.py", line
> 940,
> >> in __create_execution_context
> >> self._handle_dbapi_exception(e, kwargs.get('statement', None),
> >> kwargs.get('parameters', None), None, None)
> >> File "/usr/lib/pymodules/python2.6/sqlalchemy/engine/base.py", line
> 931,
> >> in _handle_dbapi_exception
> >> raise exc.DBAPIError.instance(statement, parameters, e,
> >> connection_invalidated=is_disconnect)
> >> OperationalError: (OperationalError) asynchronous connection attempt
> >> underway None [{}]
> >>
> >> Do I understand that when using the gevent, I have to use async option?
> >
> > I leave that to psycopg guys to answer.
>
> An async connection it is not usable just after it's been created:
> it's in an half baked state where poll() returns POLL_READ or
> POLL_WRITE until the client and the server have completed the
> asynchronous login handshake. The protocol must be pushed by the
> client as described in
> <http://initd.org/psycopg/docs/advanced.html#asynchronous-support>.
> The wait loop should be somewhere after the connection creation and
> before the usage: I don't know if the libraries you are using allow
> that.
>
> Note that if you are using gevent and not twisted you may use the
> green mode instead
> <
> http://initd.org/psycopg/docs/advanced.html#support-to-coroutine-libraries
> >,
> that provides you async connections whose behaviour is entirely
> equivalent to the blocking one (they block a single greenlet and
> schedule the other to go). I suspect you should try them instead of
> the raw async mode.
>
> -- Daniele
>