Re: Two weeks to feature freeze

On Fri, Jun 20, 2003 at 03:39:47PM -0700, Dann Corbit wrote:

> We (at CONNX Solutions Inc.) have a formal release procedure that
> includes many tens of thousands of automated tests using dozens of
> different platforms. [...]
>
> If there is no procedure for PostgreSQL of this nature, then there
> really needs to be. I am sure that MySQL must have something in place
> like that. Their "Crash-Me" test suite shows (at least) that they have
> put a large effort into testing.

The regression testing suite in Postgres is one of the things that
impresses me about this software. It's very rare that a change is even
committed to the main tree if a single regression test doesn't pass.
When it does, a proper fix is quickly put in or the change reverted.

It's even rare that patches with regression failures get posted in
pgsql-patches. Regression tests are a very handy tool for contributors
to check that their work is "safe". It's considered good practice to
submit new tests when there's new functionality in a patch.

There probably isn't such a gigantic effort like the one you describe,
but there certainly _is_ a testing procedure. There's probably room for
improvement, of course, but we don't want the tests to take a full week
to complete, IMHO.

It would be nice to have a system which could receive a patch and
compile and verify that it passes the tests before it goes to Bruce's
queue; or compile on multiple platforms to check for portability
problems, for example.

--
Alvaro Herrera (<alvherre[a]dcc.uchile.cl>)
"Uno puede defenderse de los ataques; contra los elogios se esta indefenso"