Re: HTTP Frontend? (and a brief thought on materialized views)

On Fri, Mar 30, 2012 at 9:11 AM, Andrew Dunstan <andrew(at)dunslane(dot)net> wrote:
>
>
> On 03/30/2012 11:41 AM, Robert Haas wrote:
>>
>> On Fri, Mar 30, 2012 at 10:55 AM, Dobes Vandermeer<dobesv(at)gmail(dot)com>
>>  wrote:
>>>
>>> Well, in our case HTTP is a clear win (but not replacement) and SPDY a
>>> potential one (even as a replacement).  Even if SPDY is not widely
>>> adopted
>>> it could still replace FEBE if there's a clear advantage to using it, I
>>> don't know enough to make the call right now.
>>
>> I can see that there are some advantages to having an HTTP interface
>> to the database, but I think throwing our existing protocol out the
>> window or relegating it to the status of a second-class citizen would
>> be foolish.
>
>
>
> Right, I can't imagine it happening. And I wouldn't really be keen to add an
> alternative protocol either.
>
> I could imagine a client which presented a SPDY interface to the world and
> translated it into standard calls, possibly via libpq.

Any enhancement here that can't be used with libpq via, say, drop-in
.so seems unworkable to me, and that's why any solution that is
basically proxying to the database is basically a non-starter outside
the very earliest prototyping stages. The tuple scanning and protocol
semantics can and even should remain the same, especially at first.

What I really want is for a mechanism to push FEBE messages down into
a transport in a extendable way (not unlike SSL, except this time
sensitive to FEBE message boundaries), taking unmodified libpq
binaries (but with a hook, of course...) into consideration.

> It's well to remember that we are not a green fields project here.

Perhaps not, which is a good reason to not put cart before horse --
I've tried to be very careful to suggest that real useful
caching/proxy/routing software needs to appear that can work,
unmodified, with hypothetical FEBE-over-SPDY communications, and
accomplish some interesting use cases, and also very careful to
suggest this is a *transport*-level feature. It think would be fair
to demand: "Okay, when Apache or NGINX or a credible embedded web
server in some language can accomplish something useful for us if we
absorb this complexity, let's put this on the table".

To have to write continuously write poolers, cachers and redirectors
(for any reason: HA, partitioning, or even just clean-looking domain
names...) is a burden on community resources. The criteria for success
is to expand the number of projects attacking these problems and
decrease the cost of writing one, and I think demanding a straw-man
implementation of, say, a query cacher or pooler and seeing how that
implementation feels is not unreasonable along with accepting the
burden of maintenance.

Robert Haas Wrote:
>> HTTP is a non-trivial protocol that tends to impose lots
>> of escaping and de-escaping overhead which is unnecessary for people
>> who just want to connect to the database and run queries.  I can
>> completely understand that someone might want the ability to do GET
>> /db/table/pk and have that return an answer very, very quickly, by
>> bypassing the usual parser and planner and just firing off an
>> index-scan and returning the results as JSON or somesuch.  But I think
>> it would be a serious mistake to assume that GET /q?q=myquery is going
>> to come out better than what we have now in the general case.

I don't think this holds for SPDY, which is the only way I even
entertain thoughts of doing this. I want FEBE, at least at first and
probably even then for quite a long while, just with framing and
headers (during start-up) that other software will understand.

Control frames are like this:

+----------------------------------+
|C| Version(15bits) | Type(16bits) |
+----------------------------------+
| Flags (8) | Length (24 bits) |
+----------------------------------+
| Data |
+----------------------------------+

Data frames are like this:

+----------------------------------+
|C| Stream-ID (31bits) |
+----------------------------------+
| Flags (8) | Length (24 bits) |
+----------------------------------+
| Data |
+----------------------------------+

The (XML, not HTML, I read this with "view source", or try "curl") RFC
draft is here:

http://mbelshe.github.com/SPDY-Specification/draft-mbelshe-spdy-00.xml

--
fdr