Re: spinlocks: generalizing "non-locking test"

On Sun, Oct 17, 2004 at 11:16:50PM +1000, Neil Conway wrote:
> Currently, the assembly for TAS() on x86 does a non-locking test before
> using an atomic operation to attempt to acquire the spinlock:
>
> __asm__ __volatile__(
> " cmpb $0,%1 \n"
> " jne 1f \n"
> " lock \n"
> " xchgb %0,%1 \n"
> "1: \n"
> : "+q"(_res), "+m"(*lock)
> :
> : "memory", "cc");
>
> The reason this is a good idea is that if we fail to immediately acquire
> the spinlock, s_lock() will spin SPINS_PER_DELAY times in userspace
> calling TAS() each time before going to sleep. If we do an atomic
> operation for each spin, this generates a lot more bus traffic than is
> necessary. Doing a non-locking test (followed by an atomic operation to
> acquire the spinlock if appropriate) is therefore better on SMP systems.
>
> Currently x86 is the only platform on which we do this -- ISTM that all
> the other platforms that implement spinlocks via atomic operations could
> benefit from this technique.
>
> We could fix this by tweaking each platform's assembler to add a
> non-blocking test, but there might be an easier way. Rather than
> modifying platform-specific assembler, I believe this C sequence is
> equivalent to the non-locking test:
>
> volatile slock_t *lock = ...;
>
> if (*lock == 0)
> TAS(lock);
>
> Because the lock variable is volatile, the compiler should reload it
> from memory for each loop iteration. (If this is actually not a
> sufficient non-locking test, please let me know...)
>
> We could add a new s_lock.h macro, TAS_INNER_LOOP(), whose default
> implementation would be:
>
> #define TAS_INNER_LOOP(lock) \
> if ((*lock) == 0) \
> TAS(lock);
>
> And then remove the x86-specific non-locking test from TAS.
>
> Comments?
>
> -Neil
>

Steve Hemminger had this to say:

The linux kernel code is:
#define UNLOCKED 1

...
while (atomic_dec(lock) != 0) {
do {
rep_nop();
} while(*lock != UNLOCKED);
}

To do the equivalent thing in postgres would mean

while(TAS(lock)) {
do {
rep_nop();
} while (*lock);

The point is do the locking test first (assume success is possible)
if that doesn't work spin without doing locked operations and make
sure and do the rep; nop; for the hyperthreaded CPU's