On Mon, May 5, 2008 at 5:33 PM, Craig James <craig_james(at)emolecules(dot)com> wrote:
>
> (It also reinforces what the pgbench man page says: Short runs aren't
> useful. Even these two-minute runs have a lot of variability. Before I
> turned off AutoVacuum, the variability was more like 50% between runs.)
I'd suggest a couple things for more realistic tests. Run the tests
much longer, say 30 minutes to an hour. Crank up your scaling factor
until your test db is larger than memory. Turn on autovacuum, maybe
raising the cost / delay factors so it doesn't affect performance too
negatively. And lastly tuning the bgwriter so that checkpoints are
short and don't interfere too much.
My guess is if you let it run for a while, you'll get a much more
reliable number.