resolver timeout error at moderate load

[Vinay Y S]

Hi,
I'm testing performance of nginx as reverse proxy where it selects the
upstream server by composing a dns name based on the incoming request's host
header. For example if the request is for username.domain.com/images, the
upstream server is username-images.internal. This username-images.internal
is looked up in a internal dns server which is configured using the resolver
directive.

When I hit nginx with requests that use about 30000 different hostnames
using http_load tool with 1600 parallel requests, I see this error:

2010/03/24 15:12:50 [error] 11650#0: *2260409
user12-images.nginxtest.comcould not be resolved (110: Operation timed
out), client: 10.10.65.221,
server: *.nginxtest.com, request: "GET /10kb HTTP/1.0", host: "
user12.nginxtest.com"

I've tried with resolver_timeout from 1 second to 10 seconds. This error
doesn't happen with lesser number of dns record set (like 100 different
usernames). But happens consistently with 30k users. I'm sure there is no
issues with the dns server itself because I've run dnsperf on the dns server
and it can do 20000 queries/second with max time being less than a second.

Also, isn't nginx supposed to cache the dns records for 300 seconds?

Any help in resolving this issue is greatly appreciated.

Thanks,
Vinay
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://nginx.org/pipermail/nginx/attachments/20100325/cae9a0f2/attachment-0001.html>