Re: another autovacuum scheduling thread

> > I suspect the most likely area the new prioritisation order could
> > cause issues is from the lack of randomness. Will multiple workers
> > working into the same database be more likely to bump into each other
> > somehow in a bad way? Maybe that's a good area to focus testing.
>
> I agree that lack of randomness could cause problems, but I don't see
> how it could cause regressions, because the current system isn't
> random, either. Even if the order of pg_class is unpredictable, it may
> (depending on the workload) not change very much from one day to the
> next.
>
> > Yeah partly, but mostly I just really doubt that this matters that
> > much. It's been said on this thread already that prioritisation isn't
> > as important as the autovacuum-configured-to-run-too-slowly issue, and
> > I agree with that. I just find it hard to believe that the highly
> > volatile pg_class order has been just perfect all these years and that
> > sorting by percentage-over-threshold-desc will make things worse
> > overall. There was mention that pg_catalog tables are first in
> > pg_class, but I don't really agree with that as if I create some new
> > tables on a fresh database, I see those getting lower ctids than any
> > pg_catalog table. The space for that is finite, but there's no
> > shortage of other reasons for user tables to become mentioned in
> > pg_class before catalogue tables as the database gets used. I see that
> > table_beginscan_catalog() uses SO_ALLOW_SYNC too, so there's an extra
> > layer of randomness from sync scans. I don't recall any complaints
> > from the order autovacuum works on tables, so, to me, it just seems
> > strange to think that the volatile order of pg_class just happened to
> > be right all these years. I suspect what's happening is that the extra
> > bloat or stale statistics that people get as a result of the
> > pg_class-order autovacuum just gets unnoticed, ignored or attended to
> > via adjustments to the corresponding scale_factor reloption.
>
> Interesting. I don't have any real knowledge of how jumbled-up the
> order of pg_class is on real production systems, and I agree that if
> the answer is "it's usually quite jumbled up" then that is good news
> for this patch. In any case, I'm not trying to say that prioritization
> is an intrinsically bad idea, because I don't believe that. What I'm
> trying to say is that there's a limited number of ways for this patch
> to make things worse

What I have not been able to prove from my tests is that the processing
order of tables by autovacuum will actually make things any better or any
worse. My tests have been short 30 minute tests that count how many
vacuum cycles tables with various DML activity and sizes received.
I have not found much difference. I am also not sure how valuable
these short-duration tests are either.

On the field is where the real test occurs and it may be discovered that
the new strategy improves the majority of the cases, and there may also
be cases where the existing strategy is somehow better. Having the
ability to go back to the existing behavior seems like the best way we
can roll this out and learn over time.

These may be the only two strategies we will ever need, or we may find out
that a third strategy in which individual tables are assigned a prioritization
score will also be useful.

--
Sami