Re: distributed performance testing

On Monday 22 August 2005 13:13, Jim C. Nasby wrote:
> On Sat, Aug 13, 2005 at 06:29:58PM -0400, Tom Lane wrote:
> > Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > > Incidentally, use of a different SCM system might well make
> > > constructing test sets more simple. Imagine, say, in SVN, you would
> > > create a branch called "test-set-yyyy-mm-dd" or some such, make your
> > > changes there, add a test script under some well known name, and commit
> > > the branch.
> >
> > This seems a pretty unconvincing argument for SVN ... we could perfectly
> > well do that in CVS, no?
>
> In any case, I think it probably makes more sense to specify tests as
> 'pull from CVS as of this date/tag and then (optionally) apply these
> patches'. It doesn't seem to make sense to clutter up CVS just to be
> able to run performance tests.
>
> In any case, I agree. I've been wondering if it makes sense to setup a
> result repository for dbt* where people running dbt tests could submit
> results (along with machine config, etc). ISTM that having that would be
> beneficial on it's own, and we could then build an additional framework
> for pushing desired tests out to a set of machines.
>
> Of course we could use pgbench for this instead of dbt*, but ISTM that
> dbt is a better choice since it's useful for a broader set of people.
> The downside is it requires dbt, but that doesn't seem to be a major
> issue. Also, using dbt means we can test different use cases (dbt2 ~=
> TPC-C, dbt3 ~= TPC-H, etc), while pgbench is just a single benchmark.

And there is always http://pgfoundry.org/projects/tpc-w-php/ for a ~= TPC-W
workload.

--
Darcy Buskermolen
Wavefire Technologies Corp.

http://www.wavefire.com
ph: 250.717.0200
fx: 250.763.1759