Re: .ready and .done files considered harmful

+ * This .ready file is created out of order, notify archiver to perform
+ * a full directory scan to archive corresponding WAL file.
+ */
+ StatusFilePath(archiveStatusPath, xlog, ".ready");
+ if (stat(archiveStatusPath, &stat_buf) == 0)
+ PgArchEnableDirScan();

We may want to call PgArchWakeup() after setting the flag.

+ * Perform a full directory scan to identify the next log segment. There
+ * may be one of the following scenarios which may require us to perform a
+ * full directory scan.
...
+ * - The next anticipated log segment is not available.

I wonder if we really need to perform a directory scan in this case.
Unless there are other cases where the .ready files are created out of
order, I think this just causes an unnecessary directory scan every
time the archiver catches up.

+ * Flag to enable/disable directory scan. If this flag is set then it
+ * forces archiver to perform a full directory scan to get the next log
+ * segment.
+ */
+ pg_atomic_flag dirScan;

I personally don't think it's necessary to use an atomic here. A
spinlock or LWLock would probably work just fine, as contention seems
unlikely. If we use a lock, we also don't have to worry about memory
barriers.

Nathan