Re: Create shorthand for including all extra tests

On Mon, Sep 04, 2023 at 04:30:31PM -0400, Tom Lane wrote:
> Noah Misch <noah(at)leadboat(dot)com> writes:
> > On Mon, Sep 04, 2023 at 08:16:44PM +0200, Daniel Gustafsson wrote:
> >> On 4 Sep 2023, at 17:01, Tom Lane <tgl(at)sss(dot)pgh(dot)pa(dot)us> wrote:
> >>> I think this is a seriously bad idea. The entire point of not including
> >>> certain tests in check-world by default is that the omitted tests are
> >>> security hazards, so a developer or buildfarm owner should review each
> >>> one before deciding whether to activate it on their machine.
>
> > Other than PG_TEST_EXTRA=wal_consistency_checking, they have the same hazard:
> > they treat the loopback interface as private, so anyone with access to
> > loopback interface ports can hijack the test. I'd be fine with e.g.
> > PG_TEST_EXTRA=private-lo activating all of those. We don't gain by inviting
> > the tester to review the tests to rediscover this common factor.
>
> Yeah, I could live with something like that from the security standpoint.
> Not sure if it helps Nazir's use-case though. Maybe we could invent
> categories that can be used in place of individual test names?
> For now,
>
> PG_TEST_EXTRA="needs-private-lo slow"
>
> would cover the territory of "all", and I think it'd be very seldom
> that we'd have to invent new categories here (though maybe I lack
> imagination today).

I could imagine categories for filesystem bytes and RAM bytes. Also, while
needs-private-lo has a bounded definition, "slow" doesn't. If today's one
"slow" test increases check-world duration by 1.1x, we may not let a
100x-increase test use the same keyword.

If one introduced needs-private-lo, the present spelling of "all" would be
"needs-private-lo wal_consistency_checking". Looks okay to me. Doing nothing
here wouldn't be ruinous, of course.