Re: Rewriting the test of pg_upgrade as a TAP test

Andres,

* Andres Freund (andres(at)anarazel(dot)de) wrote:
> On 2017-04-05 10:50:19 -0400, Stephen Frost wrote:
> > Probably because the point was brought up that the regression tests for
> > pg_upgrade spend a bunch of time doing something which, ultimately,
> > don't actually add any real value. Yes, there are bits of the core
> > regression tests that currently add value over what we have through
> > other approaches, but that's not where the bulk of running those tests
> > go.
>
> Create a separate patch [& thread] about that, don't conflate the
> topics. I'm very much in favor of this rewrite, I'm very much not in
> favor of only using some targeted testsuite. By combining two
> independent changes, you're just making it less likely that anything
> happens.

I've made it clear, I thought, a couple of times that I agree with the
rewrite and that we should move forward with it. Nothing on this
sub-thread changes that. It's also registered in the 2017-07
commitfest, so I wouldn't think that there's a risk of it being
forgotten or that we need to cut off all discussion about what may
change between now and July that would be relevant to this patch.

> > We don't look at the gin index after the upgrade in the current
> > pg_upgrade testing, so I don't see why you feel it's at all valuable.
>
> It's be trivial to add a VACUUM to the point where analyze_new_cluster
> is currently run. And I've previously run more manual tests. Is that
> perfect - no, definitely not.

Being trivial doesn't mean it's something we're actually doing today.

Given that we aren't actually changing anything in the index during a
same-version pg_upgrade, nor are we changing the code that's run by
that VACUUM, I'm curious just what we're ending up testing that's
different from just restarting the existing cluster and running a new
VACUUM.

Thanks!

Stephen