Re: pgsql: Attempt to fix unstable regression tests, take 2

David Rowley <drowley(at)postgresql(dot)org> writes:
> Attempt to fix unstable regression tests, take 2

That still isn't working, and I'm starting to think that we're
blaming the messenger. To wit, what we are seeing here is that
these tests are unstable ... and that's a bug of the tests, not of
any adjustments of autovacuum timing. I notice that at least
most of the deltas are in test cases that are only a couple weeks
old, so they don't have a lot of track record saying that they
are stable.

Specifically, I notice that in the last couple of failures involving
tests on "mcv_lists", we are considering stats on a 5000-row table.
With the standard setting default_statistics_target = 100, ANALYZE
will take a 3000-row random sample, meaning that its results are
*inherently* not 100% reproducible. In the past we have avoided
that being a problem for the regression tests by dint of not
depending on exact estimation results, but this test is designed
to do precisely that.

I think what needs to happen here is to revert these changes
you've made, and instead either reduce the size of the test table
or increase default_statistics_target within this test, so that
ANALYZE's sample includes the whole table and is thereby completely
stable. (I'd tend to favor the former approach if possible, just
to keep a lid on test runtime.)

Or we could decide that these tests are a bad idea and toss them
completely; but I bet Tomas will object.

regards, tom lane