Re: How can we make beta testing better?

From: Josh Berkus <josh(at)agliodbs(dot)com>
To: Rod Taylor <rod(dot)taylor(at)gmail(dot)com>
Cc: Pg Hackers <pgsql-hackers(at)postgresql(dot)org>
Subject: Re: How can we make beta testing better?
Date: 2014-04-17 23:42:21
Message-ID: 535066DD.8090908@agliodbs.com
Views: Raw Message | Whole Thread | Download mbox | Resend email
Thread:
Lists: pgsql-hackers

On 04/15/2014 09:53 PM, Rod Taylor wrote:
> A documented beta test process/toolset which does the following would help:
> 1) Enables full query logging
> 2) Creates a replica of a production DB, record $TIME when it stops.
> 3) Allow user to make changes (upgrade to 9.4, change hardware, change
> kernel settings, ...)
> 4) Plays queries from the CSV logs starting from $TIME mimicking actual
> timing and transaction boundaries
>
> If Pg can make it easy to duplicate activities currently going on in
> production inside another environment, I would be pleased to fire a couple
> billion queries through it over the next few weeks.
>
> #4 should include reporting useful to the project, such as a sampling of
> queries which performed significantly worse and a few relative performance
> stats for overall execution time.

So we have some software we've been procrastinating on OSS'ing, which does:

1) Takes full query CSV logs from a running postgres instance
2) Runs them against a target instance in parallel
3) Records response times for all queries

tsung and pgreplay also do this, but have some limitations which make
them impractical for a general set of logs to replay.

What it would need is:

A) scripting around coordinated backups
B) Scripting for single-command runs, including changing pg.conf to
record data.
C) tools to *analyze* the output data, including error messages.

--
Josh Berkus
PostgreSQL Experts Inc.
http://pgexperts.com

In response to

Responses

Browse pgsql-hackers by date

  From Date Subject
Next Message Josh Berkus 2014-04-17 23:47:21 Re: assertion failure 9.3.4
Previous Message Fabrízio de Royes Mello 2014-04-17 23:38:01 Re: DISCARD ALL (Again)