Re: Resumable vacuum proposal and design overview

Galy Lee <lee(dot)galy(at)oss(dot)ntt(dot)co(dot)jp> writes:
> For example, there is one table:
> - The table is a hundreds GBs table.
> - It takes 4-8 hours to vacuum such a large table.
> - Enabling cost-based delay may make it last for 24 hours.
> - It can be vacuumed during night time for 2-4 hours.

> It is true there is no such restrict requirement that vacuum
> need to be interrupt immediately, but it should be stopped in an
> *predictable way*. In the above example, if we have to wait for the end
> of one full cycle of cleaning, it may take up to 8 hours for vacuum to
> stop after it has received stop request. This seems quit unacceptable.

I think you misunderstood what Simon means by "cycle", because you are
claiming that one cycle == one complete table VACUUM; if that were so
then what he is proposing would be exactly the status quo. What he's
proposing (which is the same thing I was going to say, until I saw he'd
beat me to it) is that you should be prepared to stop after any one
cycle of fill-work-mem-and-clean-indexes. This should not take that
long given the current physical-scan-order implementation of
btbulkdelete, especially if you don't set maintenance_work_mem too
large. Moreover, it avoids a boatload of risks associated with assuming
that a batch of TIDs you've hidden somewhere are still valid. It's not
hard to come up with scenarios where you could be discarding live tuples
because of reliance on a stale TID-list file. State that consists only
of a next-heap-page-to-scan is just a whole lot more robust.

regards, tom lane