From: | Thomas Munro <thomas(dot)munro(at)gmail(dot)com> |
---|---|
To: | Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> |
Cc: | Andrew Dunstan <andrew(dot)dunstan(at)2ndquadrant(dot)com>, David Rowley <dgrowleyml(at)gmail(dot)com>, PostgreSQL Developers <pgsql-hackers(at)lists(dot)postgresql(dot)org> |
Subject: | Re: Recording test runtimes with the buildfarm |
Date: | 2020-06-10 21:43:43 |
Message-ID: | CA+hUKG+XByDLUqdFGRva82DM5ykvT6+cG+AFw+yu4CFE2P9ruQ@mail.gmail.com |
Views: | Raw Message | Whole Thread | Download mbox | Resend email |
Thread: | |
Lists: | pgsql-hackers |
On Thu, Jun 11, 2020 at 2:13 AM Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> I have in the past scraped the latter results and tried to make sense of
> them. They are *mighty* noisy, even when considering just one animal
> that I know to be running on a machine with little else to do. Maybe
> averaging across the whole buildfarm could reduce the noise level, but
> I'm not very hopeful. Per-test-script times would likely be even
> noisier (ISTM anyway, maybe I'm wrong).
I've been doing that in a little database that pulls down the results
and analyses them with primitive regexes. First I wanted to know the
pass/fail history for each individual regression, isolation and TAP
script, then I wanted to build something that could identify tests
that are 'flapping', and work out when the started and stopped
flapping etc. I soon realised it was all too noisy, but then I
figured that I could fix that by detecting crashes. So I classify
every top level build farm run as SUCCESS, FAILURE or CRASH. If the
top level run was CRASH, than I can disregard the individual per
script results, because they're all BS.
From | Date | Subject | |
---|---|---|---|
Next Message | Tom Lane | 2020-06-10 21:56:45 | Re: Recording test runtimes with the buildfarm |
Previous Message | David Rowley | 2020-06-10 21:13:51 | Re: Recording test runtimes with the buildfarm |