Re: 8.5 release timetable, again

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Robert Haas <robertmhaas(at)gmail(dot)com>
Cc: Bruce Momjian <bruce(at)momjian(dot)us>, Peter Eisentraut <peter_e(at)gmx(dot)net>, Andrew Dunstan <andrew(at)dunslane(dot)net>, pgsql-hackers(at)postgresql(dot)org
Subject: Re: 8.5 release timetable, again
Date: 2009-08-27 14:11:24
Message-ID: 3050.1251382284@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Robert Haas <robertmhaas(at)gmail(dot)com> writes:
> ... That sounds a lot like the definition of a
> regression test suite. Of course, we have that already, but it's
> nowhere near comprehensive. Maybe we should be looking at an expanded
> test suite that runs on a time scale of hours rather than seconds.

mysql's got one of those, and I haven't noticed that it's kept their
defect rate down any. Hour-long regression suites are the sort of
thing developers won't run. Worse, regression suites are necessarily
designed to exercise only 100%-reproducible behavior.

> I don't think that any test suite is going to eliminate the need for
> beta-testing.

Precisely...

What I'd like to see is some sort of test mechanism for WAL recovery.
What I've done sometimes in the past (and recently had to fix the tests
to re-enable) is to kill -9 a backend immediately after running the
regression tests, let the system replay the WAL for the tests, and then
take a pg_dump and compare that to the dump gotten after a conventional
run. However this is quite haphazard since (a) the regression tests
aren't especially designed to exercise all of the WAL logic, and (b)
pg_dump might not show the effects of some problems, particularly not
corruption in non-system indexes. It would be worth the trouble to
create a more specific test methodology.

In short: merely making the tests bigger doesn't impress me in the
least. Focused testing on areas we aren't covering at all could be
worth the trouble.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Alvaro Herrera 2009-08-27 14:17:32 Re: Unicode UTF-8 table formatting for psql text output
Previous Message Kevin Grittner 2009-08-27 14:07:05 Re: 8.5 release timetable, again