499 error

[lsdillard]

I have a configuration that uses nginx + unicorn + rails 3.1.  I have a
load balancer in front of this, and the health checks are failing
because nginx is sending a 499 response when I try to use a rails page
for the health check.  I can see that the rails response was a 200 from
the logs, but nginx turns this into a 499.  I understand that this means
the client closed the connection, but its idle timeout is 60 seconds,
and the rails log shows consistent sub-1 second response times.  There's
a second, more frequent health check that just opens a TCP connection on
port 80 and only validates that the connection was opened.  When I use a
static page (either from rails public directory or from another
directory), everything's fine.  I cannot figure out why nginx is sending
this 499 response -- one page indicated it was because of not having the
host in the request
(http://www.docunext.com/blog/2009/09/nginx-499-error.html), but I set
the host in my nginx config to a hardcoded hostname, and still no luck. 
I am very confused.  I can paste config files if that helps.

Here's a look at the logs, both from rails and from nginx:


Rails (the Host: X.X.X.X line is added from the controller for debug
purposes and is actually the IP address of the server):

Started GET "/health_check" for 127.0.0.1 at Thu Dec 01 08:11:05 +0000
2011
  Processing by HealthChecksController#do_health_check as */*
  ^[[1m^[[35mHealthCheck Load (0.1ms)^[[0m  SELECT "health_checks".*
FROM "health_checks" LIMIT 1
  ^[[1m^[[36m (0.2ms)^[[0m  ^[[1mUPDATE "health_checks" SET "curr" =
'2011-12-01 08:11:06.000000', "prev" = '2011-12-01 08:10:35.000000',
"updated_at" = '2011-12-01 08:11:06.019519', "check_count" = 157 WHERE
"health_checks"."id" = 2^[[0m
Host: X.X.X.X
Rendered health_checks/do_health_check.html.erb within
layouts/application (0.4ms)
Completed 200 OK in 70ms (Views: 25.9ms | ActiveRecord: 0.7ms)


Started GET "/health_check" for 127.0.0.1 at Thu Dec 01 08:11:35 +0000
2011
  Processing by HealthChecksController#do_health_check as */*
  ^[[1m^[[35mHealthCheck Load (0.1ms)^[[0m  SELECT "health_checks".*
FROM "health_checks" LIMIT 1
  ^[[1m^[[36m (0.2ms)^[[0m  ^[[1mUPDATE "health_checks" SET "prev" =
'2011-12-01 08:11:06.000000', "check_count" = 158, "curr" = '2011-12-01
08:11:35.000000', "updated_at" = '2011-12-01 08:11:35.926096' WHERE
"health_checks"."id" = 2^[[0m
Host: X.X.X.X
Rendered health_checks/do_health_check.html.erb within
layouts/application (0.4ms)
Completed 200 OK in 36ms (Views: 12.9ms | ActiveRecord: 0.6ms)


nginx:

X.X.X.X - - [01/Dec/2011:07:57:05 +0000] "GET /health_check HTTP/1.1"
499 0 "-" "HealthChecker/1.0"
X.X.X.X - - [01/Dec/2011:07:57:15 +0000] "-" 400 0 "-" "-"
X.X.X.X - - [01/Dec/2011:07:57:25 +0000] "-" 400 0 "-" "-"
X.X.X.X - - [01/Dec/2011:07:57:35 +0000] "-" 400 0 "-" "-"
X.X.X.X - - [01/Dec/2011:07:57:35 +0000] "GET /health_check HTTP/1.1"
499 0 "-" "HealthChecker/1.0"

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,219411,219411#msg-219411