Skip site navigation (1) Skip section navigation (2)

Re: 8.5 release timetable, again

From: Robert Haas <robertmhaas(at)gmail(dot)com>
To: Bruce Momjian <bruce(at)momjian(dot)us>
Cc: Peter Eisentraut <peter_e(at)gmx(dot)net>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: 8.5 release timetable, again
Date: 2009-08-27 13:58:42
Message-ID: 603c8f070908270658n26ed7c44gfc52675a8a3a23e8@mail.gmail.com (view raw or flat)
Thread:
Lists: pgsql-hackers
On Wed, Aug 26, 2009 at 7:39 PM, Bruce Momjian<bruce(at)momjian(dot)us> wrote:
> Peter Eisentraut wrote:
>> Much of the delay and uncertainty during beta in my mind comes from the
>> situation that we wait for negative results and don't trust the release
>> until we have seen and fixed enough of them.  Instead of waiting for
>> concrete, positive results and producing the release with confidence.
>
> Yep, that is our dilemma.

To get positive results in which you can have confidence, you have to
know that the testing which was done actually did a reasonably good
job exercising the code in a way that would have flushed out bugs, had
any been present.  That sounds a lot like the definition of a
regression test suite.  Of course, we have that already, but it's
nowhere near comprehensive.  Maybe we should be looking at an expanded
test suite that runs on a time scale of hours rather than seconds.
Actually, didn't Peter talk about something like this at PGCon?

I don't think that any test suite is going to eliminate the need for
beta-testing.  But if we could say that we had a regression test suite
which covered X% of our code, and it passed on all Y platforms tested,
that would certainly be a confidence booster, especially for large
values of X.  What we're basically doing now is hoping that
beta-testers come up with some novel tests, and that's a bit
hit-or-miss.

Part of the question, of course, is how to build up such a regression
test suite.  One way to do it is try to add a test every time you fix
a bug, such that the new test fails on the unpatched code and passes
with the fix...  or we could have a expanded-regression-test only
commitfest during beta, or something.  I think people would be willing
to put in some work on this.  I certainly would.  I run the regression
tests frequently during development but unless I'm making system
catalog changes they rarely catch anything.  I'm not prepared to write
the whole thing myself, but I would definitely be willing to develop
and contribute some tests.

...Robert

In response to

Responses

pgsql-hackers by date

Next:From: Alvaro HerreraDate: 2009-08-27 14:03:26
Subject: Re: Unicode UTF-8 table formatting for psql text output
Previous:From: Grzegorz JaskiewiczDate: 2009-08-27 12:39:46
Subject: Re: clang's static checker report.

Privacy Policy | About PostgreSQL
Copyright © 1996-2014 The PostgreSQL Global Development Group