Re: WIP: WAL prefetch (another approach)

On Sat, Dec 12, 2020 at 1:24 AM Jakub Wartak <Jakub(dot)Wartak(at)tomtom(dot)com> wrote:
> I wanted to contribute my findings - after dozens of various lengthy runs here - so far with WAL (asynchronous) recovery performance in the hot-standby case. TL;DR; this patch is awesome even on NVMe

Thanks Jakub! Some interesting, and nice, results.

> The startup/recovering gets into CPU 95% utilization territory with ~300k (?) hash_search_with_hash_value_memcmpopt() executions per second (measured using perf-probe).

I suppose it's possible that this is caused by memory stalls that
could be improved by teaching the prefetching pipeline to prefetch the
relevant cachelines of memory (but it seems like it should be a pretty
microscopic concern compared to the I/O).

> [3] - hash_search_with_hash_value() spends a lot of time near "callq *%r14" in tight loop assembly in my case (indirect call to hash comparision function). This hash_search_with_hash_value_memcmpopt() is just copycat function and instead directly calls memcmp() where it matters (smgr.c, buf_table.c). Blind shot at gcc's -flto also didn't help to gain a lot there (I was thinking it could optimize it by building many instances of hash_search_with_hash_value of per-match() use, but no). I did not quantify the benefit, I think it just failed optimization experiment, as it is still top#1 in my profiles, it could be even noise.

Nice. A related specialisation is size (key and object). Of course,
simplehash.h already does that, but it also makes some other choices
that make it unusable for the buffer mapping table. So I think that
we should either figure out how to fix that, or consider specialising
the dynahash lookup path with a similar template scheme.

Rebase attached.