Occasional successful upstreamed requests that don't get picked up

[rcutter]

Hello, I believe I have a tuning issue with NGINX - hoping someone can point
me in the right direction!

About 1 in 60,000 requests being proxied through Kong/NGINX are timing out. 
These requests are getting to the upstreamed host and are successfully
logged in the load balancer in front of this upstreamed host.  So either
there's a network issue between that load balancer and NGINX or NGINX is
simply not able/willing to process the response.

Assuming this is an NGINX tuning issue, these are my settings (note these
hosts have 32 cores).  Traffic is not all that high, less than 10 req/sec
per instance and requests are usually satisfied in less than a second:

worker_processes auto;
worker_priority -10;
daemon off;
pid pids/nginx.pid;
error_log logs/error.log debug;

worker_rlimit_nofile 20000;

events {
    worker_connections 10000;
    multi_accept on;
}

Most all other config settings are "default" values.  There's nothing in the
Kong logs that indicate these dropped responses are being processed by Kong.
 There's no indication there aren't enough workers.  These timeouts do not
happen in clusters, they are more like singletons.

Any advice on things I should look at or diagnosis possibilities?  Thanks
very much, Ryan

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274267,274267#msg-274267