From: | Peter Eisentraut <peter_e(at)gmx(dot)net> |
---|---|
To: | Robert Haas <robertmhaas(at)gmail(dot)com> |
Cc: | Bruce Momjian <bruce(at)momjian(dot)us>, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>, Andrew Dunstan <andrew(at)dunslane(dot)net>, pgsql-hackers(at)postgresql(dot)org |
Subject: | Re: 8.5 release timetable, again |
Date: | 2009-08-27 18:54:23 |
Message-ID: | 1251399263.31175.12.camel@vanquo.pezone.net |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On tor, 2009-08-27 at 09:58 -0400, Robert Haas wrote:
> To get positive results in which you can have confidence, you have to
> know that the testing which was done actually did a reasonably good
> job exercising the code in a way that would have flushed out bugs, had
> any been present. That sounds a lot like the definition of a
> regression test suite. Of course, we have that already, but it's
> nowhere near comprehensive. Maybe we should be looking at an expanded
> test suite that runs on a time scale of hours rather than seconds.
> Actually, didn't Peter talk about something like this at PGCon?
Let's look at it this way: If I were writing a compiler, then I would
have two main test approaches. First, I would have an in-tree test
suite that compiles a bunch of example code snippets and checks that the
results are reasonable. Call that a regression test. It would be
informed by code coverage analysis and previously reported bugs.
Second, as part of my release cycle, I would have an agenda to try to
compile a large set of real programs against my new compiler version.
I would do that during the beta period. You will notice that GCC pretty
much operates that way.
We have regression tests. They could and should be expanded. That's a
developer job, and we can start working on that now. But this
discussion was about what to do during beta. And I think during beta
you want to test PostgreSQL against a large set of real applications.
But we could try to clarify how to actually do that in an organized way.
Now, if you want to improve the regression tests, I would suggest going
through the commits since 8.4beta and since 8.4.0 final release and ask
how these problems could have been prevented or caught earlier. I
suppose a test suite for WAL might be part of the answer, but a closer
analysis might be insightful.
From | Date | Subject | |
---|---|---|---|
Next Message | Robert Haas | 2009-08-27 19:04:20 | Re: 8.5 release timetable, again |
Previous Message | Werner Echezuria | 2009-08-27 18:45:19 | return a set of records |