Default server showing incorrectly for site homepage only

[EricHarth]

Hi All

I have a rather strange issue that appears to have only started in the last
few days (no changes have been made in the last few days) . I'll try and
describe it as best I can but please let me know if I need to post
particular configs etc.

I have a setup with an nginx load balancer in front of 2 backend nginx web
servers running on CentOS 6.3. This site has been running fine up until now
but recently when visiting the home page it has been showing me the page I
created as the default site on the webserver (it is a discrete notice that
says the host header has not been recognised on the webserver). When you
visit a sub-page e.g. domain.com/about-us it is fine but visiting domain.com
shows the default site. It appears that this problem can be solved by
disabling the caching on the load balancer for this particular site. As soon
as the cache is not used, the homepage shows correctly.

The cache on the load balancer is held in a RAM drive mapped to /cache with
a total available size of 256Mb. The cache configuration is as follows

proxy_cache_path                /cache levels=1:2 keys_zone=app-cache:30m
max_size=126m inactive=10m;
proxy_temp_path                 /cache/tmp;
proxy_buffer_size               8192;
proxy_max_temp_file_size        1m;

I reduced the total size of the cache last night as I have a feeling it was
causing another problem I saw recently where we came under extreme load, the
cache filled to the max allowed size of 256Mb but that left no room on the
mount point for the temp path to buffer files from the upstreams and started
serving empty files to clients.

I've looked in the error log for the particular site and found a number of
instances of 

2013/01/03 05:47:24 [crit] 22889#0: *352983 pwrite() "/cache/tmp/0000028264"
failed (28: No space left on device) while reading upstream, client:
121.58.173.7, server: www.domain.com, request: "GET /images/slider-plus.gif
HTTP/1.1", upstream: "http://10.0.100.193:80/images/slider-plus.gif", host:
"www.domain.com", referrer: "http://www.domain.com/example-page"

A df -h shows the /cache mount to only be using 1% of the available space at
the moment, so I'm not sure why I should be still getting this error.

The nginx version on the load balancer is 1.0.11 (built from source)
The nginx version on the web server is 1.0.15 (installed from package from
epel)

I realise that nginx is in need of updating, I only realised this yesterday.
If this is the problem then I can bring forward the plans to update it, but
it'd be useful to have confirmation of what this problem might be and how it
can be solved.

Please feel free to request any extra info needed to help diagnose this
situation.

Many thanks

Eric

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,234716,234716#msg-234716