Re: Testing

On Friday 28 November 2003 13:41, Jussi Mikkola wrote:
> Okay. The tests. Is it so, that if I make a new feature, I also make a
> test for it?

Depends on the feature really.

> What if that new feature runs without errors, but is very
> slow, uses plenty of memory etc. ? Can it leak memory? Is there a test
> for all new features? What happens, if there is no test for a feature?
> Is there a link from a bug to change tests?

If the feature is slow/incomplete etc then it will only be used by those that
really need that feature. Badly written code wont make it into the system
because the core developers will refuse to commit it.
It is not easy to define a test for all features - do you test to see if it
works or it won't break? What about testing how all features interact? How do
you define what is/is not a feature? Having said that, you've seen the
regression tests and they check the standard operations work as they should.

> Well, when I look at the release history, and for example 7.3, there was
> 7.3.4 that came out before 7.4. So there are not very many releases.
> That means, that there can't be very many serious bugs left around. Many
> bugs would mean that there would need to be minor releases quite often.

If there were many bugs, or bugs that were common then PostgreSQL wouldn't be
in use. If you want to see the process of releasing a new version check the
mailing list archives for pgsql-hackers and -bugs. A new release goes through
a number of beta-tests and release candidates before being finalised.

> Usually it is not just luck, that a product works. But I would like to
> tell, that there is a reason, and that also the future releases are very
> reliable ;-) Certainly, I would not like to tell, that as a user, you
> are the tester... But of course, it is different, if we say, that as a
> user, you have the ability to test the software, and verify that it
> works properly ;-)

With PG, some of the users are also the testers. Some of the users are the
developers, some are advocates, some build add-on tools and some help out on
the mailing lists.
Of course, you don't have to do any of this, you could just run 7.3.x and wait
until 7.4.1 or 7.4.2 is out, by which time any bugs should be pretty obscure.
What I try to do is test the development version of my applications against
new releases while sticking with the last stable release on my customers'
machines.

--
Richard Huxton
Archonet Ltd