Re: [HACKERS] kqueue

On Sat, Jan 25, 2020 at 11:29 AM Thomas Munro <thomas(dot)munro(at)gmail(dot)com> wrote:
> On Thu, Jan 23, 2020 at 9:38 AM Rui DeSousa <rui(at)crazybean(dot)net> wrote:
> > Here is two charts comparing a patched and unpatched system.
> > These systems are very large and have just shy of thousand
> > connections each with averages of 20 to 30 active queries concurrently
> > running at times including hundreds if not thousand of queries hitting
> > the database in rapid succession. The effect is the unpatched system
> > generates a lot of system load just handling idle connections where as
> > the patched version is not impacted by idle sessions or sessions that
> > have already received data.
>
> Thanks. I can reproduce something like this on an Azure 72-vCPU
> system, using pgbench -S -c800 -j32. The point of those settings is
> to have many backends, but they're all alternating between work and
> sleep. That creates a stream of poll() syscalls, and system time goes
> through the roof (all CPUs pegged, but it's ~half system). Profiling
> the kernel with dtrace, I see the most common stack (by a long way) is
> in a poll-related lock, similar to a profile Rui sent me off-list from
> his production system. Patched, there is very little system time and
> the TPS number goes from 539k to 781k.

If there are no further objections, I'm planning to commit this sooner
rather than later, so that it gets plenty of air time on developer and
build farm machines. If problems are discovered on a particular
platform, there's a pretty good escape hatch: you can define
WAIT_USE_POLL, and if it turns out to be necessary, we could always do
something in src/template similar to what we do for semaphores.