Re: Fwd: Re: Running make check-world in buildfarm (was Re: [COMMITTERS] pgsql: Use SASLprep to normalize passwords for SCRAM authentication.)

On 04/08/2017 10:11 AM, Andrew Dunstan wrote:
>
>
> -------- Forwarded Message --------
> Subject: Re: [HACKERS] Running make check-world in buildfarm (was Re:
> [COMMITTERS] pgsql: Use SASLprep to normalize passwords for SCRAM
> authentication.)
> Date: Sat, 8 Apr 2017 07:05:54 -0700
> From: Andres Freund <andres(at)anarazel(dot)de>
> To: Michael Paquier <michael(dot)paquier(at)gmail(dot)com>, Andrew Dunstan
> <andrew(at)dunslane(dot)net>
> CC: Heikki Linnakangas <hlinnaka(at)iki(dot)fi>, Tom Lane
> <tgl(at)sss(dot)pgh(dot)pa(dot)us>, pgsql-hackers <pgsql-hackers(at)postgresql(dot)org>
>
>
>
> On 2017-04-08 23:01:06 +0900, Michael Paquier wrote:
> > On Sat, Apr 8, 2017 at 7:33 PM, Heikki Linnakangas <hlinnaka(at)iki(dot)fi> wrote:
> > > Hmm. It looks like none of the buildfarm members are running the
> > > authentication tests. Nor recovery tests, nor subscription tests. We're
> > > missing a trick here, at least some of the buildfarm members really ought to
> > > run "make check-world", we're missing a lot of coverage otherwise.
> >
> > I recall that Andrew has been favoring as much as possible one folder
> > path per test series in the buildfarm client (perhaps to keep the
> > tests separated and have the logs easier to analyze?). I would not
> > mind much if this is replaced by a pure make check-world, which is
> > what most of the serious hackers do, or at least a make check running
> > from src/test to save us a lot of maintenance pain.
>
> I think it's partially knowing which target failed, and which
> regression.diffs to display. If we were able to revamp check-world so
> it outputs a list of targets the regression machinery were able to run
> individually, it'd probably help?
>

Yes, I don't want just to run check-world.

I am aware of a few test sets that need to be added, and I'm planning on
doing that this weekend, in fact. Specifically: recovery, subscription,
authentication and SSL. Peter Eisentraut raised this with me about a
week ago.

Instead of just adding targets to check-world, perhaps we need to look
at what we can do so that the buildfarm client can discover what checks
it might run and run them, just as we specify test schedules for pg_regress.

cheers

andrew

--

Andrew Dunstan https://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services