[PATCH] Added asm ISB as asm pause for ngx_cpu_pause() for aarch64

[julio.suarez at foss.arm.com]

# HG changeset patch
# User Julio Suarez <julio.suarez at arm.com>
# Date 1701877879 21600
#      Wed Dec 06 09:51:19 2023 -0600
# Node ID 53d289b8676fc678ca90e02ece174300a3631f79
# Parent  f366007dd23a6ce8e8427c1b3042781b618a2ade
Added asm ISB as asm pause for ngx_cpu_pause() for aarch64

For aarch64 (A.K.A. Arm64), ngx_cpu_pause() evaluates to empty by the
GCC preprocessor. This results in an empty for loop in the lock
check code in ngx_rwlock.c/ngx_spinlock.c (a bug).
Thus, on Arm CPUs, there is no wait between checks of a lock.

When a delay like the PAUSE that is there for x86 is added,
there is a 2-5% increase in the number of requests/sec Arm CPUs
can achieve under high load.

Currently, options for a spin lock delay element
are YIELD and ISB. YIELD is assembled into a NOP for most Arm CPUs.
YIELD is for Arm implementations with SMT.
Most Arm implementations are single core - single thread (non-SMT).
Thus, this commit uses ISB (Instruction Barrier Sync) since it
will provide a slightly longer delay than a NOP.

Other projects that implement spin locks have used ISB as
the delay element which has yielded performance gains as well.

Last, ISB is not an architecturally defined solution
for short spin lock delays.
A candidate for a short delay solution is the WFET
instruction (Wait For Event with Timeout).
However, there aren't any Arm implementations in the market that
support this instruction yet. When that occurs,
WFET can be tested as a replacement for ISB. Until then,
ISB will do.

diff -r f366007dd23a -r 53d289b8676f src/os/unix/ngx_atomic.h
--- a/src/os/unix/ngx_atomic.h	Tue Nov 14 15:26:02 2023 +0400
+++ b/src/os/unix/ngx_atomic.h	Wed Dec 06 09:51:19 2023 -0600
@@ -66,6 +66,8 @@
 
 #if ( __i386__ || __i386 || __amd64__ || __amd64 )
 #define ngx_cpu_pause()             __asm__ ("pause")
+#elif ( __aarch64__ )
+#define ngx_cpu_pause()             __asm__ ("isb" ::: "memory")
 #else
 #define ngx_cpu_pause()
 #endif