Re: 8.2 is 30% better in pgbench than 8.3

From: Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us>
To: Greg Smith <gsmith(at)gregsmith(dot)com>
Cc: PostgreSQL-development <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: 8.2 is 30% better in pgbench than 8.3
Date: 2007-07-22 17:47:44
Message-ID: 2222.1185126464@sss.pgh.pa.us
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

Greg Smith <gsmith(at)gregsmith(dot)com> writes:
> I'd want a set of 3 at each configuration because even with longer runs,
> you occasionally get really odd results. Until you have 3 it can be
> unclear which is the weird one.

Yeah, pgbench results are notoriously unrepeatable. One issue is that
the first run after pgbench -i sees conditions a lot different from
subsequent runs (no free space in tables, indexes are more tightly
packed than they will be later, etc). The recently added option to
initialize the tables with a selected fillfactor might help here,
but personally I've not experimented with it.

There seems to be also some of the "good average but bad worst case"
behavior that Josh and others have pointed out in bigger benchmarks.
I've always assumed this was due to checkpointing (and autovac if
enabled). If your test run isn't long enough to cover one full
checkpoint cycle then the results will be quite variable depending
on whether it included a checkpoint or not. 8.3 might alleviate
this effect to some extent.

regards, tom lane

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Florian G. Pflug 2007-07-22 17:58:05 Re: Full page images in WAL & Cache Invalidation
Previous Message Tom Lane 2007-07-22 17:37:55 Re: Full page images in WAL & Cache Invalidation