Re: Quality and Performance

Andrew Dunstan <andrew(at)dunslane(dot)net> writes:
> Josh Berkus wrote:
>> ... DW operations aren't
>> really testable without 18 hours to generate data ... but we could test a
>> lot of things.

> Performance isn't just about humungous DW apps.

Indeed. I think the real take-home lesson from these past few days'
discussion is that *any* particular view of performance is going to
miss things that don't affect that case, but do affect somebody else.

What I find most worrisome about the notion of setting up a
performance-farm is that it will encourage us to optimize with blinkers
on --- that is, that we will consider only the specific cases measured
by whatever tests are included in the farm, and will happily pessimize
other cases. We can ameliorate that a bit if we can get a sufficiently
wide variety of test cases, but it will always be a concern. And
dogmatic positions like "only cases involving terabytes of data are
worth testing" are definitely not going to help.

regards, tom lane