using nginx to cache file serving

[mojiz]

Hi
We are using nginx & SATA disks on several of our file servers.However after
hitting 12MBytes/s of port speed (server nic is a 1gbps), the server gets
down to it's knees. SSH becomes very slow/unresponsive and our users
complain about slow speeds and dropped connections. Our users/downloaders
have very slow speed (like 512kbits/s)  so there are lots of users
downloading from each server. I thought the problem was from using SATA disk
and the bottleneck was the HDD IO speed.So we ordered a new server with SAS
hdds & RAID0, the performance has been great and our server is serving about
20MBytes/s, (same provider) 
However I'm still not satisfied. 
1.Is there any configuration options I'm missing? I'm using the default
config.
2. Both servers rams are fully utilized and the memory is 100% filled, When
I stop nginx the memory is still used up but why? (there is no other service
running on the server)
If this is the nginx cache, why it's not emptied when I exit/kill  nginx
processes?
3. My most important question is, Is it possible/wise that I use the SAS
server as a rev-proxy/cache to serve the downloads from the SATA server? My
idea is when a user connects to the SAS server to download a file, The SAS
server requests the file from the SATA server(using the internal connection
in datacenter), caches it on it's fast disk, then serves the file to my slow
downloader. This way the SATA HDD doesn't have to seek to the file location
each time and since there are lots of downloaders this happens a lot.

Sorry for the long message and thank you for your time
Mojiz

Posted at Nginx Forum: http://forum.nginx.org/read.php?2,245244,245244#msg-245244