[PATCH] Reduce LWLockWaitListLock() cache-line contention with adaptive spin reads

Hi Hackers,

The attached patch reduces cache-line contention in LWLockWaitListLock()'s spin-wait loop on large x86_64 systems. It does not change the locking primitive; it only changes how often spinning waiters re-read the shared lock state adaptively according to the contention situation on the fly, plus a small tweak to the spin-delay floor.

Problem:
On large x86_64 machines (e.g.: Intel Xeon, 2 sockets), pgbench TPC-B throughput was limited at high core and high client counts. We observed excessive time spent in LWLockWaitListLock() under high concurrency due to contention on the lock semantic. Perf profiles
show this is due to excessive cache-line bouncing between different cores and sockets. We believe the better solution for this is to reduce contention adaptively. According to this idea, we drafted a patch and validated on server CPU as below.

Approach:
1. In LWLockWaitListLock(), re-read lock->state only once every
'spins_per_lock_read' spins instead of on every spin. The interval
is adaptive: it grows (up to 256) under heavy contention and
shrinks back toward 1 otherwise.
2. In s_lock.c, lower MIN_DELAY_USEC from 1000us to 100us, which
shortens the initial spin-delay back-off.

Benchmarks (pgbench, TPC-B-like, prepared protocol)
Intel Xeon, 60 cores, SMT-on, 2 sockets (240 vCPUs):

Clients Baseline Optimized Speedup
8 27878 28246 1.01
16 42851 44072 1.03
32 53893 54312 1.01
64 68807 72838 1.06
120 66825 74037 1.11
240 58585 75577 1.29
500 49208 74868 1.52
800 46982 74330 1.58
1000 44945 71436 1.59

Summary:
- Low client counts (<= core count): no measurable impact. Throughput is
unchanged within noise (1.01x-1.06x at 8-64 clients), so lightly loaded
systems are unaffected.
- High client counts (>= 120 clients): significant improvement. The patch
delivers 1.11x-1.59x at 120-1000 clients.

Thanks,
Baohong Min
Intel DCG Software

Authors: Min, Baohong <baohong(dot)min(at)intel(dot)com>
Jin, Jun < jun(dot)i(dot)jin(at)intel(dot)com>
Kim, Andrew <andrew(dot)kim(at)intel(dot)com>