Bug/suggestion: large_client_header_buffers default too small. Error Codes should be better for overly large headers

[Andrew Cholakian]

I hope this is the right place to post bugs and feature suggestions.
nginx has been a joy to work with, but the way it handles large
headers is strange.

What I'd like to propose is having requests with headers with single
lines larger than large_client_header_buffers respond with a status of
414 rather than  400. Additionally, large_client_header_buffers should
default to a larger value,  double the platform's page size, to bring
up it up to an 8k minimum to match the largest cookie size in a
mainstream browser (IE 8) which maxes out at 5117 bytes by my
calculations.

I recently ran into this where very large (4.7kib) cookies from IE
were causing requests to my nginx server to fail with 400 Bad Request
errors. The underlying problem--besides the erroneously large
cookie--was a too  small large_client_header_buffers value. According
to the nginx docs
(http://wiki.nginx.org/NginxHttpCoreModule#large_client_header_buffers):
 "The request line can not be bigger than the size of one buffer, if
the client send a bigger header nginx returns error "Request URI too
large" (414). The longest header line of request also must be not more
than the size of one buffer, otherwise the client get the error "Bad
request" (400)."

The fact that it was a 400 error made it quite hard to debug, as the
cause was rather ambiguous, and only a small subset of clients, due to
app logic, had the swollen cookies. I ended up finding a particularly
cooperative user and getting a TCP dump to ascertain the problem.
Obviously, getting a user to agree to letting someone remote in and
run wireshark is highly unusual, and it is my guess that nginx users
who encounter this problem in the future are more likely to abandon
nginx than go to this length to figure out the issue. An error like
this should really provide more helpful info to debug on the server
side.

I've run some tests on Firefox 3.5, Chrome 3, and IE 8 to see what
their largest cookie sizes are:
Firefox 3.5.5:          4096 Bytes
IE 8.0.6001.18828:  5117 Bytes
Chrome 3.0.195.33: 4131 Bytes

These numbers were generated by this code, running on my home system,
a Windows Vista 64 bit machine (all browsers are 32 bit):

var data = '';
var last_len = -1;
while (last_len < document.cookie.length) {
  data += 'x';
  last_len = document.cookie.length;
  document.cookie = data;
}
document.write('Max Size: ' + last_len + ' Bytes');

I'm temporarily hosting it here:
http://www.andrewvc.com/cookietest.html for convenience if anyone else
wants to test their browser.

---
Andrew Cholakian
http://www.andrewvc.com