nginx as reverse caching proxy for CentOS repositories

[Wouter Schoot]

Dear list,

In an attempt to create an offline copy for my homenetwork to quickly 
install CentOS I wanted to use nginx as a reverse caching proxy so that 
nginx stores all the files the that the CentOS installer requests after 
the first time.

However, after most of the startup files (product.img, stage2.img etc) 
are downloaded and cached succesfully, it bails on the chkconfig rpm 
(which happens to be the first file it actually downloads for the 
install, from the CentOS/ folder.

I'm running nginx/0.8.34, compiled with a standard ./configure (only 
using the prefix, so the rest is left as default).

The following relevant changes to the default configuration were made:

proxy_cache_path /storage/cache/proxy_temp/ levels=1:2 keys_zone=one:10m 
inactive=100d max_size=1000M;
proxy_temp_path /storage/cache/;

         location / {
	    proxy_pass        http://mirror.oxilion.nl;
             proxy_cache             one;
             proxy_cache_key "$scheme$host$request_uri";
             proxy_cache_valid       200  100d;
         }

As far as I could google, this seemed only fair. Basic caching works as 
a charm, but it somehow chokes on the chkconfig rpm. However, when I use 
my proxy_pass mirror (and doing so, bypass nginx), it works like a 
charm. So, the mirror is up2date and up&running.

I've tried emptying my cache several times, and it doesn't matter.

This will be unable to troubleshoot without some debug. I've enabled 
caching debug and traced the (requesting) HTTP headers

T 192.168.1.113:33652 -> 192.168.1.104:80 [AP]
   GET /centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm 
HTTP/1.1..Accept-Encoding: identity..Range: bytes=440-25087..Connection: 
close..Host: 192.168.1.104..User-agent:
    urlgrabber/3.1.0 yum/3.2.22....

Accesslog:

192.168.1.113 - - [13/Mar/2010:19:16:05 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
143352 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
133216 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
123238 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
162614 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
152040 "-" "urlgrabber/3.1.0 yum/3.2.22"
192.168.1.113 - - [13/Mar/2010:19:16:06 +0100] "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" 200 
128872 "-" "urlgrabber/3.1.0 yum/3.2.22"


It tries to redownload the file several times and seems to give up after 
a few failed attempts.

Cache log:

***14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:12 +0100 MISS Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:12 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 MISS Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"
***14/Mar/2010:13:56:13 +0100 HIT Cache-Control: - Expires: - "GET 
/centos/5.4/os/i386/CentOS/chkconfig-1.3.30.1-2.i386.rpm HTTP/1.1" (200) 
"urlgrabber/3.1.0 yum/3.2.22"

The file does exists, and downloads succesfully when I simply feed this 
link to firefox or any other tool.

Anaconda (the CentOS installer) gives me a 
"chkconfig-1.3.30.1-2.i386.rpm cannot be opened. This is due to a 
missing file, a corrupt package, or corrupt media." in the installer 
screen, with no other options than retry and reboot.

For what it's worth; I've also tried nginx-0.7.65 with a comparable 
config, but to no avail.

Any hints or pointers I might have missed? I've tried several 
configoptions, but none of them seem to help. If you have a comparable 
setup, I'd love to see the relevant config parts.

Thanks,

Wouter