Socket connection failures on 1.6.1~precise

[Maxim Dounin]

Hello!

On Tue, Sep 02, 2014 at 11:00:10AM -0500, Jon Clayton wrote:

> I'm trying to track down an issue that is being presented only when I run
> nginx version 1.6.1-1~precise.  My nodes running 1.6.0-1~precise do not
> display this issue, but freshly created servers are getting floods of these
> socket connection issues a couple times a day.
> 
> /connect() to unix:/tmp/unicorn.sock failed (11: Resource temporarily
> unavailable) while connecting to upstream/
> 
> The setup I'm working with is nginx proxying requests to a unicorn socket
> powered by a ruby app.  As stated above, the error is NOT present on nodes
> running 1.6.0-1~precise, but any newly created node gets the newer
> 1.6.1-1~precise package installed and will inevitably have that error.
> 
> All settings from nodes running 1.6.0 appear to be the same as newly created
> nodes on 1.6.1 in terms of sysctl settings, nginx settings, and unicorn
> settings.  All package versions are the same except for nginx.  When I
> downgraded one of the newly created nodes to nginx 1.6.0 using the nginx ppa
> (ref:
> https://launchpad.net/~nginx/+archive/ubuntu/stable), the error was not
> present.
> 
> Is there any advice, direction, or similar issue experienced that someone
> else might be able to help me track this down?

Just some information:

- In nginx itself, the difference between 1.6.0 and 1.6.1 is fairy 
  minimal.  The only change affecting http is one code line added
  in the 400 Bad Request handling code 
  (see http://hg.nginx.org/nginx/rev/b8188afb3bbb).

- The message suggests that backend's backlog is full.  This can 
  easily happen on load spikes and/or if a backend is overloaded, 
  and usually unrelated to the nginx itself.

-- 
Maxim Dounin
http://nginx.org/