Debugging 504 Gateway Timeout and its actual cause and solution

[Maxim Dounin]

Hello!

On Wed, Jan 06, 2016 at 02:56:43AM -0500, Keyur wrote:

> Thanks Richard & itpp2015 for your response.
> 
> Further update :
> 
> There are 2 cases :
> 
> 1. 504 @ 120 seconds coming with below mentioned error :
> 
> 2016/01/05 03:50:54 [error] 1070#0: *201650845 upstream timed out (110:
> Connection timed out) while connecting to upstream, client: 66.249.74.99,
> server: x.x.x.x, request: "GET /some/url HTTP/1.1", upstream:
> "fastcgi://127.0.0.1:9000", host: "example.com"

This means that nginx failed to connect to your backend server in 
time.  This can happen in two basic cases:

- network problems (unlikely for localhost though); e.g., this can 
  happen if you have a statefull firewall configured between nginx 
  and there aren't enough states.

- backend is overloaded and doesn't accept connections fast 
  enough;

The latter is more likely, and usually happens when using 
Linux.  Try watching your backend listen socket queue (something 
like "ss -nlt" should work on Linux) and/or try switching on 
net.ipv4.tcp_abort_on_overflow sysctl to see if it's the case.

> 2. 504 @ 300 seconds coming with below mentioned error :
> 
> 2016/01/05 00:51:43 [error] 1067#0: *200656359 upstream timed out (110:
> Connection timed out) while reading response header from upstream, client:
> 115.112.161.9, server: 192.168.12.101, request: "GET /some/url HTTP/1.1",
> upstream: "fastcgi://127.0.0.1:9000", host: "example.com"

The message suggests the backend failed to respond in time to a 
particular request.  Depending on the request this may be either 
some generic problem (i.e., the backend is overloaded) or a 
problem with handling of the particular request.  Try debugging 
what happens on the backend.

-- 
Maxim Dounin
http://nginx.org/