Re: Quality and Performance

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 28 Nov 2007 00:15:48 -0500
Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:

> Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> > Josh Berkus wrote:
> >> ... DW operations aren't
> >> really testable without 18 hours to generate data ... but we could
> >> test a lot of things.
>
> > Performance isn't just about humungous DW apps.
>
> Indeed. I think the real take-home lesson from these past few days'
> discussion is that *any* particular view of performance is going to
> miss things that don't affect that case, but do affect somebody else.
>
> What I find most worrisome about the notion of setting up a
> performance-farm is that it will encourage us to optimize with
> blinkers on --- that is, that we will consider only the specific
> cases measured by whatever tests are included in the farm, and will
> happily pessimize other cases. We can ameliorate that a bit if we
> can get a sufficiently wide variety of test cases, but it will always
> be a concern. And dogmatic positions like "only cases involving
> terabytes of data are worth testing" are definitely not going to help.

Well I certainly agree with that, especially considering that although
we do have installations with that type of data, the percentage is
microscopic compared to those that don't.

I think it may be interested to host a series of different test from
pgbench, odbcbench, dbt2, custom scripts etc... to provide trending for
a particular host. That way if one host does 50tps continuously and
then changes one way or the other, there is at least a flag...

Sincerely,

Joshua D. Drake

- --

=== The PostgreSQL Company: Command Prompt, Inc. ===
Sales/Support: +1.503.667.4564 24x7/Emergency: +1.800.492.2240
PostgreSQL solutions since 1997 http://www.commandprompt.com/
UNIQUE NOT NULL
Donate to the PostgreSQL Project: http://www.postgresql.org/about/donate
PostgreSQL Replication: http://www.commandprompt.com/products/

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.6 (GNU/Linux)

iD8DBQFHTP3iATb/zqfZUUQRAgQ3AJ9N71Xgl+O4H4dr/IFagzjxu0HvAwCfZVkB
ReQmGxacCqilRstLGhoqBU4=
=03BG
-----END PGP SIGNATURE-----