Re: Occasional successful upstreamed requests that don't get picked up

[Peter Booth]

Ryan,





What is the topology of the system that you are describing?





You mention kong/nginx, an upstream host, a load balancer, clients  ...





Are the load balancers hardware or software devices? 


Is kong nginx simply forwarding to a load-balancer VIP that fronts multiple upstream systems?


Are there any firewalls or intrusion detection systems also in the mix?


Are your clients remote? internet based?


Are your nginx instances running on physical hosts, virtual machines, cloud based VMs?


Are your nginx instances and your upstream system s in the same datacenter or some distance apart?





Peter


 


On May 16, 2017, at 12:57 PM, rcutter <nginx-forum at forum.nginx.org> wrote:






Hello, I believe I have a tuning issue with NGINX - hoping someone can point
me in the right direction!

About 1 in 60,000 requests being proxied through Kong/NGINX are timing out. 
These requests are getting to the upstreamed host and are successfully
logged in the load balancer in front of this upstreamed host. So either
there's a network issue between that load balancer and NGINX or NGINX is
simply not able/willing to process the response.

Assuming this is an NGINX tuning issue, these are my settings (note these
hosts have 32 cores). Traffic is not all that high, less than 10 req/sec
per instance and requests are usually satisfied in less than a second:

worker_processes auto;
worker_priority -10;
daemon off;
pid pids/nginx.pid;
error_log logs/error.log debug;

worker_rlimit_nofile 20000;

events {
 worker_connections 10000;
 multi_accept on;
}

Most all other config settings are "default" values. There's nothing in the
Kong logs that indicate these dropped responses are being processed by Kong.
 There's no indication there aren't enough workers. These timeouts do not
happen in clusters, they are more like singletons.

Any advice on things I should look at or diagnosis possibilities? Thanks
very much, Ryan

Posted at Nginx Forum: https://forum.nginx.org/read.php?2,274267,274267#msg-274267

_______________________________________________
nginx mailing list
nginx at nginx.org
http://mailman.nginx.org/mailman/listinfo/nginx
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.nginx.org/pipermail/nginx/attachments/20170516/4973d27b/attachment-0001.html>