Re: Timeout control within tests

(I pushed the main patch as f2698ea, on 2022-03-04.)

On Fri, Feb 18, 2022 at 06:41:36PM -0800, Noah Misch wrote:
> On Fri, Feb 18, 2022 at 10:26:52AM -0500, Tom Lane wrote:
> > Noah Misch <noah(at)leadboat(dot)com> writes:
> > > On Thu, Feb 17, 2022 at 09:48:25PM -0800, Andres Freund wrote:
> > >> Meson's test runner has the concept of a "timeout multiplier" for ways of
> > >> running tests. Meson's stuff is about entire tests (i.e. one tap test), so
> > >> doesn't apply here, but I wonder if we shouldn't do something similar?
> >
> > > Hmmm. It is good if the user can express an intent that continues to make
> > > sense if we change the default timeout. For the buildfarm use case, a
> > > multiplier is moderately better on that axis (PG_TEST_TIMEOUT_MULTIPLIER=100
> > > beats PG_TEST_TIMEOUT_DEFAULT=18000). For the hacker use case, an absolute
> > > value is substantially better on that axis (PG_TEST_TIMEOUT_DEFAULT=3 beats
> > > PG_TEST_TIMEOUT_MULTIPLIER=.016666).
> >
> > FWIW, I'm fairly sure that PGISOLATIONTIMEOUT=300 was selected after
> > finding that smaller values didn't work reliably in the buildfarm.
> > Now maybe 741d7f1 fixed that, but I wouldn't count on it. So while I
> > approve of the idea to remove PGISOLATIONTIMEOUT in favor of using this
> > centralized setting, I think that we might need to have a multiplier
> > there, or else we'll end up with PG_TEST_TIMEOUT_DEFAULT set to 300
> > across the board. Perhaps the latter is fine, but a multiplier seems a
> > bit more flexible.
>
> The PGISOLATIONTIMEOUT replacement was 2*timeout_default, so isolation suites
> would get 2*180s=360s. (I don't want to lower any default timeouts, but I
> don't mind raising them.) In a sense, PG_TEST_TIMEOUT_DEFAULT is a multiplier
> with as many sites as possible multiplying it by 1. The patch has multiples
> at two code sites.

Here's the PGISOLATIONTIMEOUT replacement patch. I waffled on whether to
back-patch. Since it affects only isolation suite testing, only on systems
too slow for the default timeout, it's not a major decision. I currently plan
not to back-patch, since slow systems that would have wanted a back-patch can
just set both variables.