Re: datachecksums: handle invalid and dropped databases during enable

Thanks for the report and analysis!

I managed to reproduce it locally by SIGSTOPping the backend just
before the DROP, it fails exactly after 5 seconds with the same error.

The session terminated by DROP DATABASE ... WITH (FORCE) still holds
the temporary table the test uses to hold the worker in place. During
exit, RemoveTempRelationsCallback drops the temp table in a new
transaction, and the commit of that transaction calls XLogFlush. That
flush queues on WALWriteLock behind the WAL the checksum workers just
generated, and CountOtherDBBackends only waits 50 * 100ms, so on slow
storage DROP DATABASE gives up before the flush
completes.

0001 fixes the test. The background session now uses
synchronous_commit = off, so its exit-time commit does not flush at
all, and the test checkpoints right before the DROP so the few
remaining exit-time WAL inserts cannot queue behind a large backlog.

While reproducing this I found another issue: When DROP fails this
way, it has already terminated the checksums worker, so the launcher
sees a dead worker and a database which still exists, and that cancels
the whole operation and reverts the checksum state to off:

ERROR: database "dropmeforce" is being accessed by other users
...
ERROR: data checksums failed to get enabled in all databases, aborting

The same happens if an administrator simply terminates the worker with
pg_terminate_backend.

0002 makes the launcher retry the database in this situation. The
worker records in shared memory when it is terminated by SIGTERM, and
if the database still exists the launcher starts a new worker for it
instead of aborting.
A worker failing still aborts the operation as before, and canceling
the launcher also still works.