Re: Streaming replication and WAL archive interactions

Here is v7 after one more round of self-review. Also I want to add
more context here: we used v4 in production for ~3-4 months, and it's
mostly fine, except exceptions (see cascading replication bug, WAL
removal on checkpoint bug in 056 tap etc).

I also want to state that archive mode=always is not really a cure for
us, since it would increase RPS to our WAL archive storage in 2.5-3x.

It's mostly refactoring, I also deleted this id-condition from
v7-0001. It is never reached there (see XLogArchivingActive).

```
reshke(at)reshke:~/postgres-dev/src/test/recovery$ git diff
../../backend/postmaster/pgarch.c
diff --git a/src/backend/postmaster/pgarch.c b/src/backend/postmaster/pgarch.c
index 7137d110322..38900b3d004 100644
--- a/src/backend/postmaster/pgarch.c
+++ b/src/backend/postmaster/pgarch.c
@@ -379,15 +379,6 @@ pgarch_ArchiverCopyLoop(void)
{
char xlog[MAX_XFN_CHARS + 1];

- /*
- * In shared archive mode during recovery, the archiver doesn't archive
- * files. The primary is responsible for archiving, and the walreceiver
- * marks files as .done when the primary confirms archival. After
- * promotion, the archiver starts working normally.
- */
- if (XLogArchiveMode == ARCHIVE_MODE_SHARED && RecoveryInProgress())
- return;
-
/* force directory scan in the first call to pgarch_readyXlog() */
arch_files->arch_files_size = 0;
```

Currently I'm looking at how streaming from an archive should behave
in the case of a shared archive (What happens when WAL streaming is
terminated and the startup process starts to read the WAL file from
the archive?). Currently I'm struggling with a reliable way to detect
archive streaming. Maybe I will share a POC on this shortly

Best regards,
Kirill Reshke