Re: New IndexAM API controlling index vacuum strategies

On Wed, Mar 17, 2021 at 7:55 PM Peter Geoghegan <pg(at)bowt(dot)ie> wrote:
> Attached patch series splits everything up. There is now a large patch
> that removes the tupgone special case, and a second patch that
> actually adds code that dynamically decides to not do index vacuuming
> in cases where (for whatever reason) it doesn't seem useful.

Attached is v4. This revision of the patch series is split up into
smaller pieces for easier review. There are now 3 patches in the
series:

1. A refactoring patch that takes code from lazy_scan_heap() and
breaks it into several new functions.

Not too many changes compared to the last revision here (mostly took
things out and put them in the second patch).

2. A patch to remove the tupgone case.

Severa new and interesting changes here -- see below.

3. The patch to optimize VACUUM by teaching it to skip index and heap
vacuuming in certain cases where we only expect a very small benefit.

No changes at all in the third patch.

We now go further with removing unnecessary stuff in WAL records in
the second patch. We also go further with simplifying heap page
vacuuming more generally.

I have invented a new record that is only used by heap page vacuuming.
This means that heap page pruning and heap page vacuuming no longer
share the same xl_heap_clean/XLOG_HEAP2_CLEAN WAL record (which is
what they do today, on master). Rather, there are two records:

* XLOG_HEAP2_PRUNE/xl_heap_prune -- actually just the new name for
xl_heap_clean, renamed to reflect the fact that only pruning uses it.

* XLOG_HEAP2_VACUUM/xl_heap_vacuum -- this one is truly new, though
it's actually just a very primitive version of xl_heap_prune -- since
of course heap page vacuuming is now so much simpler.

I have also taught heap page vacuuming (not pruning) that it only
needs a regular exclusive buffer lock -- there is no longer any need
for a super-exclusive buffer lock. And, heap vacuuming/xl_heap_vacuum
records don't deal with recovery conflicts. These two changes to heap
vacuuming (not pruning) are not additional performance optimizations,
at least to me. I did things this way in v4 because it just made
sense. We don't require index vacuuming to use super-exclusive locks
[1] in any kind of standard way, nor do we require index vacuuming to
generate its own recovery conflicts (pruning is assumed to take care
of all that in every index AM, bar none). So why would we continue to
require heap vacuuming to do either of these things now?

This patch is intended to make index vacuuming and heap vacuuming very
similar. Not just because it facilitates work like the work in the
third patch -- it also happens to make perfect sense.

[1] It's true that sometimes index vacuuming uses super-exclusive
locks, but that isn't essential and is probably bad and unnecessary in
the case of nbtree. Note that GiST is fine with just an exclusive lock
today, to give one example, even though gistvacuumpage() is based
closely on nbtree's btvacuumpage() function.

--
Peter Geoghegan