Re: [HACKERS] web-based front end development

At 04:20 PM 7/29/99 +0000, Thomas Lockhart wrote:

>Hi. There is interest at work in doing some fairly simple inventory
>control using a database. We've already got Postgres up and running
>(for obvious reasons) and I was wondering if anyone had suggestions
>for a good approach to app development for brower-based user
>interfaces.

I'm fond of AOLserver and its TCL API, which has built-in pooled
db connectivity, maintaining persistent db connections with
no forking overhead. The webserver itself schedules TCL threads,
there's no CGI forking overhead as with older style Perl, etc
interfaces (of course, Apache + modPerl etc now implements a similarly
efficient interface). AOLserver's open source, GNU-licensed, and
also supports CGI and a C API. It works with many dbs, including
Postgres.

The integrated TCL API has been extended with literally dozens
of website-useful extensions to connect to the db, do selects,
do dml, get table info, send e-mail ("ns_sendmail to from message
extraheaders" - this gives a flavor of the ease in which one
gets things done in this environment), schedule TCL scripts
(I use this feature to do nightly pg_dumps), load pages from
other URLs, upload files from client browsers, etc etc etc.
If you're at all familiar with TCL you could have a few modest
pages up and running within hours of downloading the server
and the utilities package mentioned below.

An example of a website built using this combination plus
postgres 6.5.1 can be found at http://donb.photo.net/tweeterdom.

ArsDigita has a ton of code available to implement bboards,
e-commerce, bug (ticket) tracking, chat, and a bunch of other
stuff which sits on top of Oracle. I've found this stuff easy
to migrate to Postgres - the TCL scripts don't need changing
other than the actual SQL, which is mostly vanilla. The lack
of an outer join gets in the way of porting some of this code,
thus those who've discussed porting the complete ArsDigita system
are eagerly awaiting Postgres 6.6.

Other than that, the fact that Postgres implements sequences
in a way very much like Oracle with a just slightly different
syntax makes porting the SQL fairly easy. My particular
application doesn't include any of the specific modules
implemented by ArsDigita, other than that used to register
and log in visitors. But, I've poached a bunch of code from
the modules to avoid wheel-reinventing in a bunch of my own
stuff.

ArsDigita also has prototyping scripts available that quickly
builds form-based entry and edit pages from tables. This is
new, again is Oracle-based, but should be easy to move to
postgres.

They also supply a rich set of utilities which greatly
simplify using the db and forms, whether or not you use
their other tools.

All of this stuff's available from ArsDigita under the GNU
license as well, at arsdigita.com.

While not well-known, AOLserver, the ArsDigita stuff, and
Oracle lie underneath some very busy sites. Postgres 6.5.1
isn't quite robust enough to measure up to Oracle yet for
a truly busy site, but is so much better in the web environment
than 6.4 that I think it's really reasonable for modest sites,
up to a few ten thousands of hits a day at the very least.

I've tested the AOLserver+TCL+Postgres on a P200 classic
with Linux, 64 MB (running postgres -B 2000), and two IDE
disks (indices on one, datafiles on the other). On another
LAN-connected machine I fired up a bunch of browsers and
used them to run test scripts which simulated ten users
each doing four inserts a second, and two users doing
"selects" in tight TCL loops (did this in part to test
the changes which removed the unneeded logging after read-only
selects). The machine was perfectly happy performing these
tasks, not falling behind a bit. The four inserts were bundled
into a single transaction, so that's ten transactions a second
consisting of forty inserts. A lot of hits for a tiny machine,
really. Says a lot for Postgres, Linux, the web server and its
TCL API, not to mention the speed and capacity of modern
microcomputers!

If you're doing simple web applications, a straight interpreter,
be it TCL, Perl or whatever, has some advantages over a compiled
language. You make your changes, and poof! they're there, no
recompiling, relinking, etc.

Of course, I wouldn't use TCL for a large program requiring
hundreds of thousands of lines of code...but it does have a
reasonably clean syntax and the basic loop, select, if-then-else
structures and the ability to declare procedures.

The very nature of HTML imposes a structure on web programs
which is oriented around web pages, i.e. if you take input
via a form, that's a page. You process it and stuff into or
inquire from a database in the page that's the "action"
target of the form. You get to form pages via hyperlinks,
and normally they live on separate pages because that's the
natural way to present things to the user.

So the structure's typically not one imposed by the programmer's
decomposition of the problem in some abstract sense, but rather
your decomposition of the page flow through the client browser,
i.e. the design of the interface as you wish the user to see
it. That page flow design leads you around by the nose as you
develop the code underlying them (either html with embedded
TCL (.adp pages), or as I prefer scripts which write html (.tcl
pages), in AOLserver - ASP pages and ColdFusion-augmented
html are much like AOLserver .adp pages, CGI+Perl more like
the .tcl page approach).

>From the point of view of the user, this is almost cool - the
programmer's FORCED to design the user interface and the
page flow that takes the user through it. The UI - primitive
as it is - steps forward front-and-center, by necessity. One
can hope, at least, that the need to focus here leads to better
UIs, though of course HTML is so primitive that UIs are awkward
unless you get deep into client side java/javascript, which has
a lot of problems of its own.

Because of this, actual web code I've seen looks remarkably
the same regardless of the language being used. Much of
it is dominated by the writing of html, and shoving stuff
back-and-forth between pages (by URLencoding or hidden
form variables - html concepts, not Tcl or Perl or Java
concepts).

So I think it might be best to concentrate on looking for
combinations that can scale to high levels of activity, which
means avoiding CGI's forking interface (Apache and AOLserver
are both good at this), pooling DB connections (to avoid
forking off new db backends each time you connect), etc.

I'm using Linux+AOLserver+their Tcl API+Postgres+ArsDigita code
because the combination can service a lot of users on a cheap PC,
and it's all open-source/free software, not because I love Tcl
(I don't). The scripts underlying the sample site I gave above are
such relatively short and simple programs, though, that I don't mind
much what language I'm writing them in, language choice in this
case was a secondary consideration.

(given I'm a professional compiler writer, it still seems weird
to hear these words come from my mouth).

...

>Are there any options which are particularly "approachable" which
>would allow newbies to get something working if they have some
>existing code to look at?

If you're interested in the AOLserver approach, visit the ArsDigita
web site and poke around. You can download source code to all their
stuff to see what stuff look like.

- Don Baccus, Portland OR <dhogaza(at)pacifier(dot)com>
Nature photos, on-line guides, and other goodies at
http://donb.photo.net