bad performance with nginx cache

[Igor Sysoev]

On Fri, Aug 27, 2010 at 02:10:50PM +0400, Franchoze Eric wrote:

>  Hello,
> 
> On these week I tried speed up nginx work with turning on cache for graphics files specified by extentions (jpg, png, bmp, etc.) in config file. On system without any load client (web browser) loaded pages for the same time - with cache or without cache. But after installing same configuration on real server I realized the following issue:
> 
> 1. Nginx can not speed up more then 150Mbit/s is cache is no hard drive. Can speed up to 200Mbit/s if cache is on tmpfs.
> 2. Load time from client begins to be 2-3 times lager then without cache if traffic is about 150Mbit/s. This happens for all files even if they were not specified in nginx.conf.
> 
> Does anybody have successful nginx cache run with throughput 500-600Mbit/s? Does it speed up or lower end client page load speed?
> 
> My configuration is SMP machine with nginx worker per cpu.
>     proxy_cache_path  /var/www/cache2 levels=1:2 keys_zone=my-cache:512m max_size=3000m inactive=10m;
>     proxy_temp_path /var/www/cache2/tmp;
>     proxy_cache_key "$request_method|$http_if_modified_since|$http_if_none_match|$host|$request_uri";
>     
>         
>         location  ~*(site1.com|site2.com|site3.com|site4.com|site5.com|site6.com)*\.(jpe?g|gif|png|tif|svg|ico|bmp)$ {
>                         proxy_pass http://$host$request_uri;
>                         proxy_buffering on;
>                         proxy_buffers 256 4k;
>                         proxy_buffer_size 4k;
>                         proxy_cache_min_uses 1; #minimal requests to put object to cache
>                         proxy_cache my-cache;
>                         proxy_cache_valid  200 301 302 304 60m; #page ok
>                         proxy_cache_valid  404  10m; #server not found
> 
>                        log_format cache '***$time_local '
>                           '$upstream_cache_status '
>                         'Cache-Control: $upstream_http_cache_control '
>                         'Expires: $upstream_http_expires '
>                         '"$request" ($status) ';
> 
>                         access_log /var/log/nginx/cache_nginx_access_limited.log cache;
>         }
> 
> I would like to hear if there are any weak places in the cache or limitations. So maybe we can speed up it. I'm not yet very familiar with modern cache techniques but maybe we can use libvarnish to increase performance.
> I will be also ok if nginx could skip doing caching if time to fetch object from cache is bigger than download it.

What OS do you use and how much memory do you have on the host ?


-- 
Igor Sysoev
http://sysoev.ru/en/