libpq bad async behaviour

From: Daurnimator <quae(at)daurnimator(dot)com>
To: pgsql-hackers(at)postgresql(dot)org
Cc: andres(at)2ndquadrant(dot)com
Subject: libpq bad async behaviour
Date: 2015-01-09 19:57:22
Message-ID: CAEnbY+dLZBp78JYCgij7jiO1KWzf6kaD+Bkk06B9Ka57SOAX7Q@mail.gmail.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

I'm worried about libpq blocking in some circumstances; particularly
around SSL renegotiations.
This came up while writing an async postgres library for lua, I
realised that this code was dangerous:
https://github.com/daurnimator/cqueues-pgsql/blob/ee9c3fc85c94669b8128386d99d730fe93d9dbad/cqueues-pgsql.lua#L121

e.g. 1:
When a PQ connection is in non-blocking mode, PQflush returns 1, the docs say:
> wait for the socket to be write-ready and call it again
However, if the SSL layer is waiting on data for a renegotiation,
write readiness is not enough:
Waiting for POLLOUT and calling PQflush again will (untested) just
return 1 again, and continue to do so until data is recieved.
This is a busy-loop, and will block the host application.

e.g. 2:
An SSL renegiation happens while trying to receive a response.
According to 'andres' on IRC, inside of `PQisBusy` there is a busy loop:
> 14:22:32 andres You'll not see that. Even though the explanation for it is absolutely horrid.
> 14:23:32 andres There's a busy retry loop because of exactly that reason inside libpq's ssl read function whenever it hits a WANT_WRITE.
> 14:23:58 daurnimator so... libpq will block my process? :(
> 14:24:25 andres daurnimator: That case is unlikely to be hit often luckily because of the OS buffering. But yea, it's really unsatisfying.
> 14:26:06 andres daurnimator: I think this'll need a new API to be properly fixed.

One idea that came to mind if we want to keep the same api, is to hide
the socket behind an epoll file descriptor,
they always poll read ready when an fd in their set becomes ready.
I think this is also possible for kqueue on bsd, ports on solaris and
IOCP on windows.

Regards,
Daurnimator.

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Bruce Momjian 2015-01-09 20:04:19 Re: PQputCopyEnd doesn't adhere to its API contract
Previous Message David Fetter 2015-01-09 19:15:51 Re: POLA violation with \c service=